< draft-browne-sfc-nsh-kpi-stamp-07.txt   rfc8592.txt 
Network Working Group R. Browne
Internet Draft A. Chilikin Independent Submission R. Browne
Intended status: Informational Intel Request for Comments: 8592 A. Chilikin
Expires: August 2019 T. Mizrahi Category: Informational Intel
ISSN: 2070-1721 T. Mizrahi
Huawei Network.IO Innovation Lab Huawei Network.IO Innovation Lab
February 25, 2019 May 2019
A Key Performance Indicators (KPI) Key Performance Indicator (KPI) Stamping
Stamping for the Network Service Header (NSH) for the Network Service Header (NSH)
draft-browne-sfc-nsh-kpi-stamp-07
Abstract Abstract
This document describes a method of carrying Key Performance This document describes methods of carrying Key Performance
Indicators (KPIs) using the Network Service Header (NSH). This method Indicators (KPIs) using the Network Service Header (NSH). These
may be used, for example, to monitor latency and QoS marking to methods may be used, for example, to monitor latency and QoS marking
identify problems on some links or service functions. to identify problems on some links or service functions.
Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Status of This Memo
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This document is not an Internet Standards Track specification; it is
http://www.ietf.org/ietf/1id-abstracts.txt. published for informational purposes.
The list of Internet-Draft Shadow Directories can be accessed at This is a contribution to the RFC Series, independently of any other
http://www.ietf.org/shadow.html. RFC stream. The RFC Editor has chosen to publish this document at
its discretion and makes no statement about its value for
implementation or deployment. Documents approved for publication by
the RFC Editor are not candidates for any level of Internet Standard;
see Section 2 of RFC 7841.
This Internet-Draft will expire on August 25, 2019. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc8592.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document.
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction ................................................ 2 1. Introduction ....................................................2
2. Terminology ................................................. 3 2. Terminology .....................................................3
2.1. Requirement Language ................................... 3 2.1. Requirements Language ......................................3
2.2. Definition of Terms .................................... 3 2.2. Definition of Terms ........................................3
2.2.1. Terms Defined in this Document .................... 4 2.2.1. Terms Defined in This Document ......................4
2.3. Abbreviations .......................................... 4 2.3. Abbreviations ..............................................5
3. NSH KPI Stamping: An Overview ............................... 6 3. NSH KPI Stamping: An Overview ...................................6
3.1. Prerequisites .......................................... 7 3.1. Prerequisites ..............................................7
3.2. Operation ............................................. 10 3.2. Operation ..................................................9
3.2.1. Flow Selection ................................... 10 3.2.1. Flow Selection ......................................9
3.2.2. SCP Interface .................................... 10 3.2.2. SCP Interface ......................................10
3.3. Performance Considerations ............................ 11 3.3. Performance Considerations ................................11
4. NSH KPI-stamping Encapsulation ............................. 12 4. NSH KPI-Stamping Encapsulation .................................12
4.1. KPI-stamping Extended Encapsulation ................... 13 4.1. KPI-Stamping Extended Encapsulation .......................13
4.1.1. NSH Timestamping Encapsulation (Extended Mode) ... 15 4.1.1. NSH Timestamping Encapsulation (Extended Mode) .....15
4.1.2. NSH QoS-stamping Encapsulation (Extended Mode) ... 17 4.1.2. NSH QoS-Stamping Encapsulation (Extended Mode) .....17
4.2. KPI-stamping Encapsulation (Detection Mode) ........... 20 4.2. KPI-Stamping Encapsulation (Detection Mode) ...............20
5. Hybrid Models .............................................. 22 5. Hybrid Models ..................................................22
5.1. Targeted VNF Stamp .................................... 23 5.1. Targeted VNF Stamping .....................................23
6. Fragmentation Considerations ............................... 23 6. Fragmentation Considerations ...................................23
7. Security Considerations .................................... 24 7. Security Considerations ........................................24
8. IANA Considerations ........................................ 25 8. IANA Considerations ............................................24
9. Contributors ............................................... 25 9. References .....................................................25
10. Acknowledgments ........................................... 25 9.1. Normative References ......................................25
11. References ................................................ 25 9.2. Informative References ....................................25
11.1. Normative References ................................. 25 Acknowledgments ...................................................27
11.2. Informative References ............................... 26 Contributors ......................................................27
Authors' Addresses ................................................27
1. Introduction 1. Introduction
Network Service Header (NSH), as defined by [RFC8300], specifies a The Network Service Header (NSH), as defined by [RFC8300], specifies
method for steering the traffic among an ordered set of Service a method for steering traffic among an ordered set of Service
Functions (SFs) using an extensible service header. This allows for Functions (SFs) using an extensible service header. This allows for
flexibility and programmability in the forwarding plane to invoke the flexibility and programmability in the forwarding plane to invoke the
appropriate SFs for specific flows. appropriate SFs for specific flows.
NSH promises a compelling vista of operational flexibility. However, The NSH promises a compelling vista of operational flexibility.
many service providers are concerned about service and configuration However, many service providers are concerned about service and
visibility. This concern increases when considering that many service configuration visibility. This concern increases when considering
providers wish to run their networks seamlessly in 'hybrid' mode, that many service providers wish to run their networks seamlessly in
whereby they wish to mix physical and virtual SFs and run services "hybrid mode", whereby they wish to mix physical and virtual SFs and
seamlessly between the two domains. run services seamlessly between the two domains.
This document describes generic methods to monitor and debug service This document describes generic methods to monitor and debug Service
function chains in terms of latency and QoS marking of the flows Function Chains (SFCs) in terms of latency and QoS marking of the
within a service function chain. This is refered to detection mode flows within an SFC. These are referred to as "detection mode" and
and extended mode and explained in section 4. "extended mode" and are explained in Section 4.
The method described in the document is compliant with hybrid The methods described in this document are compliant with hybrid
architectures in which Virtual Network Functions (VNFs) and Physical architectures in which Virtual Network Functions (VNFs) and Physical
Network Functions (PNFs) are freely mixed in the service function Network Functions (PNFs) are freely mixed in the SFC. These methods
chain. This method also provides flexibility to monitor the also provide flexibility for monitoring the performance and
performance and configuration of an entire chain or part thereof as configuration of an entire chain or parts thereof as desired. These
desired. This method is extensible to monitoring other KPIs. Please methods are extensible to monitoring other Key Performance Indicators
refer to [RFC7665] for an architectural context for this document. (KPIs). Please refer to [RFC7665] for an architectural context for
this document.
The method described in this document is not an OAM protocol such as The methods described in this document are not Operations,
[Y.1731] or [Y.1564]. As such it does not define new OAM packet types Administration, and Maintenance (OAM) protocols such as [Y.1731]. As
or operation. Rather it monitors the service function chain such, they do not define new OAM packet types or operations. Rather,
performance and configuration for subscriber payloads and indicates they monitor the SFC's performance and configuration for subscriber
subscriber QoE rather than out-of-band infrastructure metrics. This payloads and indicate subscriber QoE rather than out-of-band
document differs from to [I-D.ippm.ioam] in the sense that it is infrastructure metrics. This document differs from [In-Situ-OAM] in
specifically tied to NSH operation and not generic in nature. the sense that it is specifically tied to NSH operations and is not
generic in nature.
2. Terminology 2. Terminology
2.1. Requirement Language 2.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
document are to be interpreted as described in BCP 14 "OPTIONAL" in this document are to be interpreted as described in
[RFC2119][RFC8174] when, and only when, they appear in all capitals, BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all
as shown here. capitals, as shown here.
2.2. Definition of Terms
This section presents the main terms used in this document. This
document makes use of the terms defined in [RFC7665] and [RFC8300].
2.2.1. Terms Defined in this Document
First Stamping Node (FSN): The first node along a service function
chain that stamps packets using KPI stamping. The FSN matches each
packet with a Stamping Controller flow based on a stamping
classification criterion such as transport 5-tuple coordinates, but
not limited to.
Last Stamping Node (LSN): The last node along a service function
chain that stamps packets using KPI stamping. From forwarding point
of view the LSN removes the NSH header and forwards the raw IP packet
to the next hop. From a control plane point of view the LSN reads all
the metadata and exports it to a system performance statistics agent
or repository. The LSN should use the NSH Service Index (SI) to
indicate if a SF was at the end of the chain. The LSN may change the
Service Path Identifier (SPI) to preconfigured value in order that
the network underlay forwards the metadata back directly to the KPI
database (KPIDB) based on this value.
Key Performance Indicator Database (KPIDB): denotes the external 2.2. Definition of Terms
storage of metadata for reporting, trend analysis, etc.
KPI-stamping: The insertion of latency-related and/or QoS-related This section presents the main terms used in this document. This
information into a packet using NSH metadata. document also makes use of the terms defined in [RFC7665] and
[RFC8300].
Flow ID: The Flow ID is a unique 16 bit identifier written into the 2.2.1. Terms Defined in This Document
header by the classifier. This allows 65536 flows to be concurrently
stamped on any given NSH service chain (SPI).
QoS-stamping: The insertion of QoS-related information into a packet First Stamping Node (FSN): The first node along an SFC that stamps
using NSH metadata. packets using KPI stamping. The FSN matches each packet with a
Stamping Controller (SC) flow based on (but not limited to) a
stamping classification criterion such as transport 5-tuple
coordinates.
Stamping Controller (SC): The SC is the central logic that decides Last Stamping Node (LSN): The last node along an SFC that stamps
what packets to stamp and how. The SC instructs the classifier on how packets using KPI stamping. From a forwarding point of view, the
to build the KPI stamp specific parts of the NSH. LSN removes the NSH and forwards the raw IP packet to the next
hop. From a control-plane point of view, the LSN reads all the
metadata (MD) and exports it to a system performance statistics
agent or repository. The LSN should use the NSH Service Index
(SI) to indicate if an SF was at the end of the chain. The LSN
may change the Service Path Identifier (SPI) to a preconfigured
value so that the network underlay forwards the MD back directly
to the KPI database (KPIDB) based on this value.
Stamp Control Plane (SCP): the control plane between the FSN and the Key Performance Indicator Database (KPIDB): Denotes the external
SC. storage of MD for reporting, trend analysis, etc.
2.3. Abbreviations KPI stamping: The insertion of latency-related and/or QoS-related
information into a packet using NSH MD.
DEI Drop Eligible Indicator Flow ID: A unique 16-bit identifier written into the header by the
classifier. This allows 65536 flows to be concurrently stamped on
any given NSH service chain.
DSCP Differentiated Services Code Point QoS stamping: The insertion of QoS-related information into a packet
using NSH MD.
FSN First Stamping Node Stamping Controller (SC): The central logic that decides what
KPI Key Performance Indicator packets to stamp and how to stamp them. The SC instructs the
classifier on how to build the parts of the NSH that are specific
to KPI stamping.
KPIDB Key Performance Indicator Database Stamping Control Plane (SCP): The control plane between the FSN and
the SC.
LSN Last Stamping Node 2.3. Abbreviations
MD Metadata DEI Drop Eligible Indicator
NFV Network Function Virtualization DSCP Differentiated Services Code Point
NFVI-PoP NFV Infrastructure Point of Presence FSN First Stamping Node
NIC Network Interface Card KPI Key Performance Indicator
NSH Network Service Header KPIDB Key Performance Indicator Database
OAM Operations, Administration, and Maintenance LSN Last Stamping Node
PCP Priority Code Point MD Metadata
PNF Physical Network Function NFV Network Function Virtualization
PNFN Physical Network Function Node NSH Network Service Header
QoE Quality of Experience OAM Operations, Administration, and Maintenance
QoS Quality of Service PCP Priority Code Point
QS QoS Stamp PNF Physical Network Function
RSP Rendered Service Path PNFN Physical Network Function Node
SC Stamping Controller QoE Quality of Experience
SCL Service Classifier QoS Quality of Service
SCP Stamp Control Plane RSP Rendered Service Path
SI Service Index SC Stamping Controller
SF Service Function SCL Service Classifier
SFC Service Function Chain SCP Stamping Control Plane
SFP Service Function Path SF Service Function
SSI Stamp Service Index SFC Service Function Chain
TC Traffic Class
TS Timestamp SI Service Index
VLAN Virtual Local Area Network SSI Stamp Service Index
TS Timestamp
VNF Virtual Network Function VLAN Virtual Local Area Network
vSwitch Virtual Switch VNF Virtual Network Function
3. NSH KPI Stamping: An Overview 3. NSH KPI Stamping: An Overview
A typical KPI stamping architecture is presented in Figure 1. A typical KPI-stamping architecture is presented in Figure 1.
Stamping Stamping
Controller Controller
| KPIDB | KPIDB
| SCP Interface | | SCP Interface |
,---. ,---. ,---. ,---. ,---. ,---. ,---. ,---.
/ \ / \ / \ / \ / \ / \ / \ / \
( SCL )-------->( SF1 )--------->( SF2 )--------->( SFn ) ( SCL )-------->( SF1 )--------->( SF2 )--------->( SFn )
\ FSN / \ / \ / \ LSN / \ FSN / \ / \ / \ LSN /
`---' `---' `---' `---' `---' `---' `---' `---'
Figure 1: Logical roles in NSH KPI Stamping
The Stamping Controller (SC) will be part of the SFC control plane Figure 1: Logical Roles in NSH KPI Stamping
architecture, but it is described separately in this document for
clarity. The SC will be part of the SFC control-plane architecture, but it is
described separately in this document for clarity.
The SC is responsible for initiating start/stop stamp requests to the The SC is responsible for initiating start/stop stamp requests to the
SCL or First Stamp Node (FSN), and also for distributing NSH stamping SCL or FSN and also for distributing the NSH-stamping policy into the
policy into the service chain via the Stamping Control Plane (SCP) service chain via the SCP interface.
interface.
The FSN will typically be part of the SCL, but again is called out as The FSN will typically be part of the SCL but is called out as a
separate logical entity for clarity. separate logical entity for clarity.
The FSN is responsible for marking NSH MD fields which tells nodes in The FSN is responsible for marking NSH MD fields; this tells nodes in
the service chain how to behave in terms of stamping at SF ingress, the service chain how to behave in terms of stamping at the SF
egress or both, or ignoring the stamp NSH metadata completely. ingress, the SF egress, or both, or ignoring the stamp NSH MD
completely.
The FSN also writes the Reference Time value, a (possibly inaccurate) The FSN also writes the Reference Time value, a (possibly inaccurate)
estimate of the current time-of-day, into the header, allowing the estimate of the current time of day, into the header, allowing the
{SPI:Flow ID} performance to be compared to previous samples for "SPI:Flow ID" performance to be compared to previous samples for
offline analysis. offline analysis.
The FSN should return an error to the SC if not synchronized to the The FSN should return an error to the SC if not synchronized to the
current time-of-day and forward the packet along the service-chain current time of day and forward the packet along the service chain
unchanged. The code and format of the error is specific to the unchanged. The code and format of the error are specific to the
protocol used between the FSN and SC; these considerations are out of protocol used between the FSN and SC; these considerations are out of
scope. scope.
SF1 and SF2 stamp the packets as dictated by the FSN and process the SF1 and SF2 stamp the packets as dictated by the FSN and process the
payload as per normal. payload as per normal.
Note 1: The exact location of the stamp creation may not be in Note 1: The exact location of the stamp creation may not be in the SF
the SF itself, as discussed in Section 3.3. itself and may be applied by a hardware device -- for
example, as discussed in Section 3.3.
Note 2: Special cases exist where some of the SFs are Note 2: Special cases exist where some of the SFs are NSH unaware.
NSH-unaware. This is covered in Section 5. This is covered in Section 5.
The Last Stamp Node (LSN) should strip the entire NSH header and The LSN should strip the entire NSH and forward the raw packet to the
forward the raw packet to the IP next hop as per [RFC8300]. The LSN IP next hop as per [RFC8300]. The LSN also exports NSH-stamping
also exports NSH stamp information to the KPI Database (KPIDB) for information to the KPIDB for offline analysis; the LSN may export the
offline analysis; the LSN may either export the stamping information stamping information of either (1) all packets or (2) a subset based
of all packets, or a subset based on packet sampling. on packet sampling.
In fully virtualized environments the LSN is likely to be co-located In fully virtualized environments, the LSN is likely to be co-located
with the SF that decrements the NSH Service Index (SI) to zero. with the SF that decrements the NSH SI to zero. Corner cases exist
Corner cases exist whereby this is not the case and is covered in where this is not the case; see Section 5.
Section 5.
3.1. Prerequisites 3.1. Prerequisites
Timestamping presents a set of prerequisites not required to QoS- Timestamping has its own set of prerequisites; however, these
Stamp. In order to guarantee metadata accuracy, all servers hosting prerequisites are not required for QoS stamping. In order to
VNFs should be synchronized from a centralized stable clock. As it is guarantee MD accuracy, all servers hosting VNFs should be
assumed that PNFs do not timestamp (as this would involve a software synchronized from a centralized stable clock. As it is assumed that
change and probable throughput performance impact) there is no need PNFs do not timestamp (as this would involve a software change and a
for them to synchronize. There are two possible levels of probable impact on throughput performance), there is no need for them
synchronization: to synchronize. There are two possible levels of synchronization:
Level A: Low accuracy time-of-day synchronization, based on Level A: Low-accuracy time-of-day synchronization, based on NTP
NTP [RFC5905]. [RFC5905].
Level B: High accuracy synchronization (typically on the order of Level B: High-accuracy synchronization (typically on the order of
microseconds), based on [IEEE1588]. microseconds), based on [IEEE1588].
Each SF SHOULD have a level A synchronization, and MAY have a level B Each SF SHOULD have Level A synchronization and MAY have Level B
synchronization. synchronization.
Level A requires each platform (including the Stamp Controller) to Level A requires each platform (including the SC) to synchronize its
synchronize its system real-time-clock to an NTP server. This is used system real-time clock to an NTP server. This is used to mark the MD
to mark the metadata in the chain, using the <Reference Time> field in the chain, using the Reference Time field in the NSH KPI stamp
in the NSH KPI-stamp header (Section 4.2). This timestamp is inserted header (Section 4.1). This timestamp is inserted into the NSH by the
to the NSH by the first SF in the chain. NTP accuracy can vary by first SF in the chain. NTP accuracy can vary by several milliseconds
several milliseconds between locations. This is not an issue as the between locations. This is not an issue, as the Reference Time is
Reference Time is merely being used as a time-of-day reference merely being used as a time-of-day reference inserted into the KPIDB
inserted into the KPIDB for performance monitoring and metadata for performance monitoring and MD retrieval.
retrieval.
Level B synchronization requires each platform to be synchronized to Level B synchronization requires each platform to be synchronized to
a Primary Reference Time Clock (PRTC) using the Precision Time a Primary Reference Clock (PRC) using the Precision Time Protocol
Protocol [IEEE1588]. A platform MAY also use Synchronous Ethernet (PTP) [IEEE1588]. A platform MAY also use Synchronous Ethernet
([G.8261], [G.8262], [G.8264]), allowing more accurate frequency [G.8261] [G.8262] [G.8264], allowing more accurate frequency
synchronization. synchronization.
If an SF is not synchronized at the moment of timestamping, it should If an SF is not synchronized at the moment of timestamping, it should
indicate its synchronization status in the NSH. This is described in indicate its synchronization status in the NSH. This is described in
more detail in Section 4. more detail in Section 4.
By synchronizing the network in this way, the timestamping operation By synchronizing the network in this way, the timestamping operation
is independent of the current Rendered Service Path (RSP). Indeed the is independent of the current RSP. Indeed, the timestamp MD can
timestamp metadata can indicate where a chain has been moved due to a indicate where a chain has been moved due to a resource starvation
resource starvation event as indicated in Figure 2, between VNF 3 and event as indicated in Figure 2, between VNF3 and VNF4 at time B.
VNF 4 at time B.
Delay Delay
| v | v
| v | v
| x | x
| x x = reference time A | x x = Reference Time A
| xv v = reference time B | xv v = Reference Time B
| xv | xv
| xv | xv
|______|______|______|______|______|_____ |______|______|______|______|______|_____
VNF1 VNF2 VNF3 VNF4 VNF5 VNF1 VNF2 VNF3 VNF4 VNF5
Figure 2: Flow performance in a service chain Figure 2: Flow Performance in a Service Chain
For QoS-stamping it is desired that the SCL or FSN be synchronized in For QoS stamping, it is desired that the SCL or FSN be synchronized
order to provide reference time for offline analysis, but this is not in order to provide a Reference Time for offline analysis, but this
a hard requirement (they may be in holdover or free-run state, for is not a hard requirement (they may be in holdover or free-run state,
example). Other SFs in the service chain do not need to be for example). Other SFs in the service chain do not need to be
synchronized for QoS-stamping operation as described below. synchronized for QoS-stamping operations, as described below.
QoS-stamping can be used to check consistency of configuration across QoS stamping can be used to check the consistency of configuration
the entire chain or part thereof. By adding all potential layer 2 and across the entire chain or parts thereof. By adding all potential
layer 3 QoS fields into a QoS sum at SF ingress or egress, this Layer 2 and Layer 3 QoS fields into a QoS sum at the SF ingress or
allows quick identification of QoS mismatches across multiple L2/L3 egress, this allows quick identification of QoS mismatches across
fields which otherwise is a manual, expert-led consuming process. multiple Layer 2 / Layer 3 fields, which otherwise is a manual,
expert-led consuming process.
| |
| |
| xy | xy
| xy x = ingress QoS sum | xy x = ingress QoS sum
| xv v = egress QoS sum | xv v = egress QoS sum
| xv y = egress QoS sum miss | xv y = egress QoS sum mismatch
| xv | xv
|______|______|______|______|______|_____ |______|______|______|______|______|_____
SF1 SF2 SF3 SF4 SF5 SF1 SF2 SF3 SF4 SF5
Figure 3: Flow QoS Consistency in a service chain Figure 3: Flow QoS Consistency in a Service Chain
Referring to Figure 3, x, v, and y are notional sum values of the QoS Referring to Figure 3, x, v, and y are notional sum values of the QoS
marking configuration of the flow within a given chain. As the marking configuration of the flow within a given chain. As the
encapsulation of the flow can change from hop to hop in terms of VLAN encapsulation of the flow can change from hop to hop in terms of VLAN
header(s), MPLS labels, DSCP(s) these values are used to compare header(s), MPLS labels, or DSCP(s), these values are used to compare
consistency of configuration from for example payload DSCP through the consistency of configuration from, for example, payload DSCP
overlay and underlay QoS settings in VLAN IEEE 802.1Q bits, MPLS bits through overlay and underlay QoS settings in VLAN IEEE 802.1Q bits,
and infrastructure DSCPs. MPLS bits, and infrastructure DSCPs.
Figure 3 indicates that at SF4 in the chain, the egress QoS marking Figure 3 indicates that, at SF4 in the chain, the egress QoS marking
is inconsistent. That is, the ingress QoS settings do not match the is inconsistent. That is, the ingress QoS settings do not match the
egress. The method described here will indicate which QoS field(s) is egress. The method described here will indicate which QoS field(s)
inconsistent, and whether this is ingress (whereby the underlay has is inconsistent and whether this is ingress (where the underlay has
incorrectly marked and queued the packet) or egress (where the SF has incorrectly marked and queued the packet) or egress (where the SF has
incorrectly marked and queued the packet. incorrectly marked and queued the packet.
Note that the SC must be aware of when a SF remarks QoS fields Note that the SC must be aware of cases when an SF re-marks QoS
deliberately and thus does not flag an issue for desired behavior. fields deliberately and thus does not flag an issue for desired
behavior.
3.2. Operation 3.2. Operation
KPI-stamping detection mode uses MD type 2 defined in [RFC8300]. This KPI-stamping detection mode uses MD Type 2 as defined in [RFC8300].
involves the SFC classifier stamping the flow at chain ingress, and This involves the SFC classifier stamping the flow at the chain
no subsequent stamps being applied, rather each SF upstream can ingress and no subsequent stamps being applied; rather, each upstream
compare its local condition with the ingress value and take SF can compare its local condition with the ingress value and take
appropriate action. Therefore detection mode is very efficient in appropriate action. Therefore, detection mode is very efficient in
terms of header size that does not grow after the classification. terms of header size that does not grow after the classification.
This is further explained in Section 4.1. This is further explained in Section 4.2.
3.2.1. Flow Selection 3.2.1. Flow Selection
The SC should maintain a list of flows within each service chain to The SC should maintain a list of flows within each service chain to
be monitored. This flow table should be in the format 'SPI:FlowID'. be monitored. This flow table should be in the format "SPI:Flow ID".
The SC should map these pairs to unique values presented as Flow IDs The SC should map these pairs to unique values presented as Flow IDs
per service chain within the NSH TLV specified in this document. The per service chain within the NSH TLV specified in this document (see
SC should instruct the FSN to initiate timestamping on flow table Section 4). The SC should instruct the FSN to initiate timestamping
match. The SC may also tell the classifier the duration of the on flow table match. The SC may also tell the classifier the
timestamping operation, either by a number of packets in the flow or duration of the timestamping operation, by either the number of
by a time duration. packets in the flow or a certain time duration.
In this way the system can monitor the performance of the all en- In this way, the system can monitor the performance of all en-route
route traffic, or an individual subscriber in a chain, or just a traffic, an individual subscriber in a chain, or just a specific
specific application or a QoS class that is used in the network. application or QoS class that is used in the network.
The SC should write the list of monitored flows into the KPIDB for The SC should write the list of monitored flows into the KPIDB for
correlation of performance and configuration data. Thus, when the correlation of performance and configuration data. Thus, when the
KPIDB receives data from the LSN it understands to which flow the KPIDB receives data from the LSN, it understands to which flow the
data pertains. data pertains.
The association of source IP to subscriber identity is outside the The association of a source IP address with a subscriber identity is
scope of this document and will vary by network application. For outside the scope of this document and will vary by network
example, the method of association of a source IP to IMSI will be application. For example, the method of association of a source IP
different to how a CPE with NAT function may be chained in an address with an International Mobile Subscriber Identity (IMSI) will
be different from how a Customer Premises Equipment (CPE) entity with
a Network Address Translation (NAT) function may be chained in an
enterprise NFV application. enterprise NFV application.
3.2.2. SCP Interface 3.2.2. SCP Interface
A Stamp Control Plane (SCP) interface is required between the SC and An SCP interface is required between the SC and the FSN or
the FSN or classifier. This interface is used to: classifier. This interface is used to:
o Query the SFC classifier for a list of active chains and flows. o Query the SFC classifier for a list of active chains and flows.
o Communicate which chains and flows to stamp. This can be a o Communicate which chains and flows to stamp. This can be a
specific {SPI:Flow ID} combination or include wildcards for specific "SPI:Flow ID" combination or can include wildcards for
monitoring subscribers across multiple chains or multiple flows monitoring subscribers across multiple chains or multiple flows
within one chain. within one chain.
o Instruct how the stamp should be applied (ingress, egress, both o Instruct how the stamp should be applied (ingress, egress, both
or specific). ingress and egress, or specific).
o Indicate when to stop stamping, either after a certain number o Indicate when to stop stamping (after either a certain number of
of packets or duration. packets or a certain time duration).
Typically SCP timestamps flows for a certain duration for trend Typically, SCP timestamps flows for a certain duration for trend
analysis, but only stamps one packet of each QoS class in a chain analysis but only stamps one packet of each QoS class in a chain
periodically (perhaps once per day or after a network change). periodically (perhaps once per day or after a network change).
Therefore, timestamping is generally applied to a much larger set of Therefore, timestamping is generally applied to a much larger set of
packets than QoS-stamping. packets than QoS stamping.
Exact specification of SCP is for further study. The exact specification of SCP is left for further study.
3.3. Performance Considerations 3.3. Performance Considerations
This document does not mandate a specific stamping implementation This document does not mandate a specific stamping implementation
method, and thus NSH KPI stamping can either be performed by hardware method; thus, NSH KPI stamping can be performed by either hardware
mechanisms, or by software. mechanisms or software.
If software-based stamping is used, applying and operating on the If software-based stamping is used, applying and operating on the
stamps themselves incur an additional small delay in the service stamps themselves incur an additional small delay in the service
chain. However, it can be assumed that these additional delays are chain. However, it can be assumed that these additional delays are
all relative for the flow in question. This is only pertinent for all relative for the flow in question. This is only pertinent for
timestamping mode, and not for QoS-stamping mode. Thus, whilst the timestamping mode, and not for QoS-stamping mode. Thus, whilst the
absolute timestamps may not be fully accurate for normal non- absolute timestamps may not be fully accurate for normal
timestamped traffic they can be assumed to be relative. non-timestamped traffic, they can be assumed to be relative.
It is assumed that the method described in this document would only It is assumed that the methods described in this document would only
operate on a small percentage of user flows. operate on a small percentage of user flows.
The service provider may choose a flexible policy in the SC to The service provider may choose a flexible policy in the SC to
timestamp a selection of user-plane every minute for example to timestamp a selection of a user plane every minute -- for example, to
highlight any performance issues. Alternatively, the LSN may highlight any performance issues. Alternatively, the LSN may
selectively export a subset of the KPI-stamps it receives, based on a selectively export a subset of the KPI stamps it receives, based on a
predefined sampling method. Of course the SC can stress test an predefined sampling method. Of course, the SC can stress-test an
individual flow or chain should a deeper analysis be required. We can individual flow or chain should a deeper analysis be required. We
expect that this type of deep analysis has an impact on the can expect that this type of deep analysis will have an impact on the
performance of the chain itself whilst under investigation. The performance of the chain itself whilst under investigation. This
impact will be dependent on vendor implementation and outside the impact will be dependent on vendor implementations and is outside the
scope of this document. scope of this document.
For QoS-stamping the method described here is even less intrusive, as For QoS stamping, the methods described here are even less intrusive,
typically multiple packets in a flow are QoS stamped periodically as typically packets are only QoS stamped periodically (perhaps once
(perhaps once per day) check one packet in a chain per QoS class. per day) to check service chain configuration per QoS class.
4. NSH KPI-stamping Encapsulation 4. NSH KPI-Stamping Encapsulation
KPI stamping uses NSH MD type 0x2 for detection of anomalies and KPI stamping uses NSH MD Type 0x2 for detection of anomalies and
extended mode for root cause analysis of KPI violations. These are extended mode for root-cause analysis of KPI violations. These are
further explained in this section. further explained in this section.
The generic NSH MD type 2 TLV for KPI Stamping is shown below. The generic NSH MD Type 2 TLV for KPI stamping is shown below.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver|O|U| TTL | Length |U|U|U|U|Type=2 | Next Protocol | |Ver|O|U| TTL | Length |U|U|U|U|Type=2 | Next Protocol |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Service Path Identifier | Service Index | | Service Path Identifier | Service Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Metadata Class | Type |U| Length | | Metadata Class | Type |U| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Variable-length KPI Metadata header and TLV(s) | | Variable Length KPI Metadata header and TLV(s) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: Generic NSH KPI Encapsulation
Relevant fields in header that the FSN must implement: Figure 4: Generic NSH KPI Encapsulation
o The O bit must not be set. Relevant fields in the header that the FSN must implement are as
follows:
o The MD type must be set to 0x2 o The O bit must not be set.
o The Metadata Class must be set to a value from the experimental o The MD type must be set to 0x2.
o The Metadata Class must be set to a value from the experimental
range 0xfff6 to 0xfffe according to an agreement by all parties to range 0xfff6 to 0xfffe according to an agreement by all parties to
the experiment. the experiment.
o Unassigned bits: all fields, marked U, are unassigned and o Unassigned bits: All fields marked "U" are unassigned and
available for future use [RFC8300] available for future use [RFC8300].
o The Type field may have one of the following values; the o The Type field may have one of the following values; the content
content of "KPI metadata" depends on the type value: of the Variable Length KPI Metadata header and TLV(s) field
depends on the Type value:
o Type = 0x01 Det: Detection * Type = 0x01 (Det): Detection
o Type = 0x02 TS: Timestamp Extended * Type = 0x02 (TS): Timestamp Extended
o Type = 0x03 QoS: QoS-stamp Extended * Type = 0x03 (QoS): QoS stamp Extended
The Type field determines the type of KPI-stamping format. The The Type field determines the type of KPI-stamping format. The
supported formats are presented in the following subsections. supported formats are presented in the following subsections.
4.1. KPI-stamping Extended Encapsulation 4.1. KPI-Stamping Extended Encapsulation
The generic NSH MD type 2 KPI-stamping header extended mode is shown The generic NSH MD Type 2 KPI-stamping header (extended mode) is
in Figure 5. This is the format for performance monitoring of service shown in Figure 5. This is the format for performance monitoring of
chain issues with respect to QoS configuration and latency. service chain issues with respect to QoS configuration and latency.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver|O|U| TTL | Length |U|U|U|U|Type=2 | Next Protocol | |Ver|O|U| TTL | Length |U|U|U|U|Type=2 | Next Protocol |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Service Path Identifier | Service Index | | Service Path Identifier | Service Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Metadata Class | Type |U| Length | | Metadata Class | Type |U| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Variable Length KPI Configuration Header | | Variable Length KPI Configuration Header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Variable Length KPI Value (LSN) | | Variable Length KPI Value (LSN) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
\ \ \ \
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Variable Length KPI Value (FSN) | | Variable Length KPI Value (FSN) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Generic KPI Encapsulation (Extended Mode)
Figure 5: Generic KPI Encapsulation (Extended Mode)
As mentioned above, two types are defined under the experimental MD As mentioned above, two types are defined under the experimental MD
class to indicate extended KPI MD: a timestamp type and a QoS-stamp class to indicate the extended KPI MD: a timestamp type and a
type. QoS-stamp type.
The KPI Encapsulation Configuration Header format is shown below. The KPI Encapsulation Configuration Header format is shown below.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|K|K|T|K|K|K|K|K| Stamping SI | Flow ID | |K|K|T|K|K|K|K|K| Stamping SI | Flow ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference Time | | Reference Time |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: KPI Encapsulation Configuration Header
The bits marked as 'K' are reserved for specific KPI type use and Figure 6: KPI Encapsulation Configuration Header
described in the corresponding subsections below.
The T bit should be set if Reference Time follows KPI Encapsulation The bits marked "K" are reserved for specific KPI type use and are
Configuration Header. described in the subsections below.
Stamping Service Index (Stamping SI) contains the Service Index used The T bit should be set if Reference Time follows the KPI
for KPI stamping and described in the corresponding subsections Encapsulation Configuration Header.
below.
The Flow ID is a unique 16 bit identifier written into the header by The SSI (Stamping SI) contains the SI used for KPI stamping and is
the classifier. This allows 65536 flows to be concurrently stamped on described in the subsections below.
any given NSH service chain (SPI). Flow IDs are not written by
subsequent SFs in the chain. The FSN may export monitored flow IDs to
the KPIDB for correlation.
Reference Time is the wall clock of the FSN, and may be used for The Flow ID is a unique 16-bit identifier written into the header by
historical comparison of SC performance. If the FSN is not Level A the classifier. This allows 65536 flows to be concurrently stamped
synchronized (see Section 3.1) it should inform the SC over the SCP on any given NSH service chain (SPI). Flow IDs are not written by
interface. The Reference Time is represented in 64-bit NTP format subsequent SFs in the chain. The FSN may export monitored Flow IDs
[RFC5905] presented in Figure 7: to the KPIDB for correlation.
Reference Time is the wall clock of the FSN and may be used for
historical comparison of SC performance. If the FSN is not Level A
synchronized (see Section 3.1), it should inform the SC over the SCP
interface. The Reference Time is represented in 64-bit NTP format
[RFC5905], as presented in Figure 7:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seconds | | Seconds |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Fraction | | Fraction |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7: NTP [RFC5905] 64-bit Timestamp Format
4.1.1. NSH Timestamping Encapsulation (Extended Mode) Figure 7: NTP 64-Bit Timestamp Format (RFC 5905)
4.1.1. NSH Timestamping Encapsulation (Extended Mode)
The NSH timestamping extended encapsulation is shown below. The NSH timestamping extended encapsulation is shown below.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver|O|C|U|U|U|U|U|U| Length |U|U|U|U|Type=2 | NextProto | |Ver|O|C|U|U|U|U|U|U| Length |U|U|U|U|Type=2 | NextProto |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Service Path ID | Service Index | | Service Path ID | Service Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Metadata Class | Type=TS(2) |U| Len | | Metadata Class | Type=TS(2) |U| Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|I|E|T|U|U|U|SSI| Stamping SI | Flow ID | |I|E|T|U|U|U|SSI| Stamping SI | Flow ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| Reference Time (T bit is set) | | Reference Time (T bit is set) |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|I|E|U|U|U| SYN | Stamping SI | Unassigned | |I|E|U|U|U| SYN | Stamping SI | Unassigned |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| Ingress Timestamp (I bit is set)(LSN) | | Ingress Timestamp (I bit is set) (LSN) |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Egress Timestamp (E bit is set)(LSN) | | Egress Timestamp (E bit is set) (LSN) |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
. . . .
. . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|I|E|U|U|U| SYN | Stamping SI | Unassigned | |I|E|U|U|U| SYN | Stamping SI | Unassigned |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| Ingress Timestamp (I bit is set) (FSN) | | Ingress Timestamp (I bit is set) (FSN) |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Egress Timestamp (E bit is set) (FSN) | | Egress Timestamp (E bit is set) (FSN) |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8: NSH Timestamp Encapsulation (Extended Mode) Figure 8: NSH Timestamp Encapsulation (Extended Mode)
The FSN KPI-stamp metadata starts with Stamping Configuration Header. The FSN KPI stamp MD starts with the Stamping Configuration Header.
This header contains the I, E, T bits and Stamp Service Index (SSI). This header contains the I, E, and T bits, and the SSI.
The I bit should be set if Ingress stamp is requested. The I bit should be set if the Ingress stamp is requested.
The E bit should be set if Egress stamp is requested. The E bit should be set if the Egress stamp is requested.
SSI field must be set to one of the following values: The SSI field must be set to one of the following values:
o 0x0 KPI-stamp mode, no Service index specified in the Stamp o 0x0: KPI stamp mode. No SI is specified in the Stamping SI field.
Service Index field.
o 0x1 KPI-stamp Hybrid mode is selected, Stamp Service Index o 0x1: KPI stamp hybrid mode is selected. The Stamping SI field
contains LSN Service index. This is used when PNFs or NSH-unaware contains the LSN SI. This is used when PNFs or NSH-unaware SFs
SFs are used at the tail of the chain. If SSI=0x1, then the value are used at the tail of the chain. If SSI=0x1, then the value in
in the type field informs the chain which SF should act as the the Type field informs the chain regarding which SF should act as
LSN. the LSN.
o 0x2 KPI-stamp Specific mode is selected, Stamp Service Index o 0x2: KPI stamp Specific mode is selected. The Stamping SI field
contains the targeted Service Index. In this case the Stamp contains the targeted SI. In this case, the Stamping SI field
Service Index field indicates which SF is to be stamped. Both indicates which SF is to be stamped. Both Ingress stamps and
ingress and egress stamps are performed when the SI=SSI on the Egress stamps are performed when the SI=SSI in the chain. For
chain. For timestamping mode, the FSN will also apply the timestamping mode, the FSN will also apply the Reference Time and
Reference Time and Ingress Timestamp. This will indicate the delay Ingress Timestamp. This will indicate the delay along the entire
along the entire service chain to the targeted SF. This method may service chain to the targeted SF. This method may also be used as
also be used as a light implementation to monitor end-to-end a light implementation to monitor end-to-end service chain
service chain performance whereby the targeted SF is the LSN. This performance whereby the targeted SF is the LSN. This is not
is not applicable to QoSStamping mode. applicable to QoS-stamping mode.
Each stamping Node adds stamping metadata which consist of Stamping Each stamping node adds stamp MD that consists of the Stamping
Reporting Header and timestamps. Reporting Header and timestamps.
The E bit should be set if Egress stamp is reported. The E bit should be set if the Egress stamp is reported.
The I bit should be set if Ingress stamp is reported. The I bit should be set if the Ingress stamp is reported.
With respect to timestamping mode, the SYN bits are an indication of With respect to timestamping mode, the SYN bits are an indication of
the synchronization status of the node performing the timestamp and the synchronization status of the node performing the timestamp and
must be set to one of the following values: must be set to one of the following values:
o In Synch: 0x00 o In synch: 0x00
o In holdover: 0x01 o In holdover: 0x01
o In free run: 0x02 o In free run: 0x02
o Out of Synch: 0x03 o Out of synch: 0x03
If the platform hosting the SF is out of synch or in free run no
timestamp is applied by the node and the packet is processed If the platform hosting the SF is out of synch or in free run, no
timestamp is applied by the node, and the packet is processed
normally. normally.
If FSN is out of synch or in free run timestamp request rejected and If the FSN is out of synch or in free run, the timestamp request is
not propagated though the chain. The FSN should inform the SC in such rejected and is not propagated through the chain. In such an event,
an event over the SCP interface. Similarly if KPIDB receives the FSN should inform the SC over the SCP interface. Similarly, if
timestamps that are out of order (i.e. a time stamp of a 'N+1' SF is the KPIDB receives timestamps that are out of order (i.e., a
in the past of a 'N' SF) it should notify SC of this condition over timestamp of an "N+1" SF is prior to the timestamp of an "N" SF), it
the SCP interface. should notify the SC of this condition over the SCP interface.
The outer service index value is copied into the stamp metadata as The outer SI value is copied into the stamp MD as the Stamping SI to
Stamping SI to help cater for hybrid chains that are a mix of VNFs help cater to hybrid chains that are a mix of VNFs and PNFs or
and PNFs or through NSH-unaware SFs. Thus, if a flow transits through through NSH-unaware SFs. Thus, if a flow transits through a PNF or
a PNF or an NSH-unaware node the delta in the inner service index an NSH-unaware node, the delta in the inner SI between timestamps
between timestamps will indicate this. will indicate this.
The Ingress Timestamp and Egress Timestamp are represented in 64-bit The Ingress Timestamp and Egress Timestamp are represented in 64-bit
NTP format. The corresponding bits (I and E) reported in the Stamping NTP format. The corresponding bits (I and E) are reported in the
Reporting Header of the node's metadata. Stamping Reporting Header of the node's MD.
4.1.2. NSH QoS-stamping Encapsulation (Extended Mode)
Packets have a variable QoS stack. That is for example the same 4.1.2. NSH QoS-Stamping Encapsulation (Extended Mode)
payload IP can have a very different stack in the access part of the
network to the core. This is most apparent in mobile networks where
for example in an access circuit we would have 2 layers of
infrastructure IP header (DSCP) - one transport-based and the other
IPsec-based, in addition to multiple MPLS and VLAN tags. The same
packet as it leaves the PDN Gateway Gi egress interface may be very
much simplified in terms of overhead and related QoS fields.
Because of this variability we need to build extra meaning into the Packets have a variable QoS stack. For example, the same payload IP
QoS headers - they are not for example all PTP timestamps of a fixed can have a very different stack in the access part of the network
length as in the case of timestamping, rather they are variable than the core. This is most apparent in mobile networks where, for
lengths and types. Also they can be changed on the underlay at any example, in an access circuit we would have an infrastructure IP
time without knowledge by the SFC system. Therefore each SF must be header (DSCP) composed of two layers -- one based on transport and
able to ascertain and record its ingress and egress QoS configuration the other based on IPsec -- in addition to multiple MPLS and VLAN
on the fly. tags. The same packet, as it leaves the Packet Data Network (PDN)
Gateway Gi egress interface, may be very much simplified in terms of
overhead and related QoS fields.
The suggested QoS type, lengths are as below. The type is 4 bits Because of this variability, we need to build extra meaning into the
long. QoS headers. They are not, for example, all PTP timestamps of a
fixed length, as in the case of timestamping; rather, they are of
variable lengths and types. Also, they can be changed on the
underlay at any time without the knowledge of the SFC system.
Therefore, each SF must be able to ascertain and record its ingress
and egress QoS configuration on the fly.
QoS Type(QT)Value Length Comment The suggested QoS Type (QT) and lengths are listed below.
IVLAN 0x01 4 Bits Ingress VLAN (PCP + DEI) QoS Type Value Length Comment
----------------------------------------------------------
IVLAN 0x01 4 Bits Ingress VLAN (PCP + DEI)
EVLAN 0x02 4 Bits Egress VLAN EVLAN 0x02 4 Bits Egress VLAN
IQINQ 0x03 8 Bits Ingress QinQ (2x PCP+DEI) IQINQ 0x03 8 Bits Ingress QinQ (2x (PCP + DEI))
EQINQ 0x04 8 Bits Egress QinQ EQINQ 0x04 8 Bits Egress QinQ
IMPLS 0x05 3 Bits Ingress Label IMPLS 0x05 3 Bits Ingress Label
EMPLS 0x06 3 Bits Egress Label EMPLS 0x06 3 Bits Egress Label
IMPLS 0x07 6 Bits 2 Ingress Labels (2x EXP) IMPLS 0x07 6 Bits Two Ingress Labels (2x EXP)
EMPLS 0x08 6 Bits 2 Egress Labels EMPLS 0x08 6 Bits Two Egress Labels
IDSCP 0x09 8 Bits Ingress DSCP IDSCP 0x09 8 Bits Ingress DSCP
EDSCP 0x0A 8 Bits Egress DSCP EDSCP 0x0A 8 Bits Egress DSCP
For stacked headers such as MPLS and 802.1ad, we extract the QoS For stacked headers such as MPLS and 802.1ad, we extract the relevant
relevant data from the header and insert into one QoS value in order to QoS data from the header and insert it into one QoS value in order to
be more efficient on packet size. Thus for MPLS, we represent both EXP be more efficient in terms of packet size. Thus, for MPLS, we
fields in one QoS value, and both 802.1p priority and drop precedence in represent both experimental bits (EXP) fields in one QoS value, and
one QoS value as indicated above. both 802.1p priority and drop precedence in one QoS value, as
indicated above.
For stack types not listed here, for example, 3 or more MPLS tags, SF For stack types not listed here (for example, three or more MPLS
would insert IMPLS/EMPLS several times with each layer in the stack tags), the SF would insert IMPLS/EMPLS several times, with each layer
indicating EXP Qos for that layer. in the stack indicating EXP QoS for that layer.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver|O|C|U|U|U|U|U|U| Length |U|U|U|U|Type=2 | NextProto=0x0 | |Ver|O|C|U|U|U|U|U|U| Length |U|U|U|U|Type=2 | NextProto=0x0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Service Path ID | Service Index | | Service Path ID | Service Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Metadata Class | Type=QoS(3) |U| Len | | Metadata Class | Type=QoS(3) |U| Len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
skipping to change at page 19, line 30 skipping to change at page 19, line 30
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| QT | QoS Value |U|U|U|E| QT | QoS Value |U|U|U|E| | QT | QoS Value |U|U|U|E| QT | QoS Value |U|U|U|E|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
. . . .
. . . .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|U|U|U|U|U|U|U|U| Stamping SI | Unassigned | |U|U|U|U|U|U|U|U| Stamping SI | Unassigned |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| QT | QoS Value |U|U|U|E| QT | QoS Value |U|U|U|E| | QT | QoS Value |U|U|U|E| QT | QoS Value |U|U|U|E|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9: NSH QoS Configuration Encapsulation (Extended Mode) Figure 9: NSH QoS Configuration Encapsulation (Extended Mode)
The encapsulation in Figure 10 is very similar to that detailed in The encapsulation in Figure 9 is very similar to the encapsulation
Section 4.1 with the following exceptions: detailed in Section 4.1.1, with the following exceptions:
- I and E bits are not required as we wish to examine the full QoS o I and E bits are not required, as we wish to examine the full QoS
stack at ingress and egress at every SF. stack at the ingress and egress at every SF.
- Syn status bits are not required. o SYN status bits are not required.
- The QT (QoS Type) and QoS value are as outlined in the table o The QT and QoS values are as outlined in the list above.
above.
- The E bit at the tail of each QoS context field indicates if this o The E bit at the tail of each QoS context field indicates if this
is the last egress QoS-stamp for a given SF. This should coincide is the last egress QoS stamp for a given SF. This should coincide
with SI=0 at the LSN, whereby the packet is truncated and the NSH with SI=0 at the LSN, whereby the packet is truncated, the NSH MD
MD sent to the KPIDB and the subscriber raw IP packet forwarded to is sent to the KPIDB, and the subscriber's raw IP packet is
the underlay next hop. forwarded to the underlay next hop.
Note: It is possible to compress the frame structure to better Note: It is possible to compress the frame structure to better
utilize the header, but this would come at the expense of crossing utilize the header, but this would come at the expense of crossing
byte boundaries. For ease of implementation, and that QoS-stamping is byte boundaries. For ease of implementation, and so that
applied on an extremely small subset of user plane traffic, we QoS stamping is applied on an extremely small subset of user-plane
believe the above structure is a pragmatic compromise between header traffic, we believe that the above structure is a pragmatic
efficiency and ease of implementation. compromise between header efficiency and ease of implementation.
4.2. KPI-stamping Encapsulation (Detection Mode) 4.2. KPI-Stamping Encapsulation (Detection Mode)
The format of the NSH MD type 2 KPI-stamping TLV (detection mode) is The format of the NSH MD Type 2 KPI-stamping TLV (detection mode) is
shown in Figure 11. shown in Figure 10.
This TLV is used for KPI anomaly detection. Upon detecting a problem This TLV is used for KPI anomaly detection. Upon detecting a problem
or an anomaly it will be possible to enable the use of KPI-stamping or an anomaly, it will be possible to enable the use of KPI-stamping
extended encapsulations, which will provide more detailed analysis. extended encapsulations, which will provide more detailed analysis.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Ver|O|U| TTL | Length |U|U|U|U|Type=2 | Next Protocol | |Ver|O|U| TTL | Length |U|U|U|U|Type=2 | Next Protocol |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Service Path Identifier | Service Index | | Service Path Identifier | Service Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Metadata Class | Type=Det(1) |U| Length | | Metadata Class | Type=Det(1) |U| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| KPI Type | Stamping SI | Flow ID | | KPI Type | Stamping SI | Flow ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Threshold KPI Value | | Threshold KPI Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Ingress KPI-stamp | | Ingress KPI stamp |
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10: Generic NSH KPI Encapsulation (Detection Mode)
The following fields are defined in the KPI TSD metadata: Figure 10: Generic NSH KPI Encapsulation (Detection Mode)
o KPI Type: determines the type of KPI-stamp that is included in The following fields are defined in the KPIDB MD:
this metadata field.
If a receiver along the path does not understand the KPI Type it
will pass the packet transparently and not drop.
The supported values of the KPI Type are:
0x0 Timestamp
0x1 QoS-stamp
o Threshold KPI Value: In the first header the SFC classifier may o KPI Type: This field determines the type of KPI stamp that is
program a KPI threshold value. This is a value that when exceeded, included in this MD. If a receiver along the path does not
requires the SF to insert the current SI value into the SI field. understand the KPI type, it will pass the packet on transparently
The KPI type is the type of KPI stamp inserted into the header as and will not drop it. The supported values of KPI Type are:
per section 9.
o Stamping SI: Service Identifier of the SF when the Threshold * 0x0: Timestamp
above is exceeded.
o Flow ID: The flow ID is inserted into the header by the SFC * 0x1: QoS stamp
o Threshold KPI Value: In the first header, the SFC classifier may
program a KPI threshold value. This is a value that, when
exceeded, requires the SF to insert the current SI value into the
SI field. The KPI type is the type of KPI stamp inserted into the
header as per Figure 10.
o Stamping SI: This is the Service Identifier of the SF when the
above threshold value is exceeded.
o Flow ID: The Flow ID is inserted into the header by the SFC
classifier in order to correlate flow data in the KPIDB for classifier in order to correlate flow data in the KPIDB for
offline analysis. offline analysis.
o Ingress KPI-stamp: The last 8 octets are reserved for the KPI- o Ingress KPI stamp: The last 8 octets are reserved for the
stamp. This is the KPI value at the chain ingress at the SFC KPI stamp. This is the KPI value at the chain ingress at the SFC
classifier. Depending on the KPI Type, the KPI-stamp either classifier. Depending on the KPI type, the KPI stamp includes
includes a timestamp or a QoS-stamp. either a timestamp or a QoS stamp. If the KPI type is Timestamp,
If the KPI Type is Timestamp, then the Ingress KPI-stamp field then the Ingress KPI stamp field contains a timestamp in 64-bit
contains a timestamp in 64-bit NTP timestamp format. If the KPI NTP timestamp format. If the KPI type is QoS stamp, then the
Type is QoS-stamp, then the format of the 64-bit Ingress KPI-stamp format of the 64-bit Ingress KPI stamp is as follows.
is as follows.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| QT | QoS Value | Unassigned | | QT | QoS Value | Unassigned |
+-+-+-+-+-+-+-+-+-+-+-+-+ + +-+-+-+-+-+-+-+-+-+-+-+-+ +
| | | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 11: QoS-stamp Format (Detection Mode)
As an example operation, say we are using KPI type 0x01 (timestamp) Figure 11: QoS-Stamp Format (Detection Mode)
when a service function (SFn) receives the packet it can compare
current local timestamp (it first checks that it is synchronized to
network PRC) with chain ingress timestamp to calculate the latency in
the chain. If this value exceeds the timestamp threshold, it then
inserts its SI and returns the NSH to the KPIDB. This effectively
tells the system that at SFn the packet violated the KPI threshold.
Please refer to figure 9 for timestamp format.
When this occurs the SFC control plane system would then invoke the As an example operation, let's say we are using KPI type 0x01
(Timestamp). When an SF (say SFn) receives the packet, it can
compare the current local timestamp (it first checks that it is
synchronized to the network's PRC) with the chain Ingress Timestamp
to calculate the latency in the chain. If this value exceeds the
timestamp threshold, it then inserts its SI and returns the NSH to
the KPIDB. This effectively tells the system that at SFn the packet
violated the KPI threshold. Please refer to Figure 8 for the
timestamp format.
When this occurs, the SFC control-plane system would then invoke the
KPI extended mode, which uses a more sophisticated (and intrusive) KPI extended mode, which uses a more sophisticated (and intrusive)
method to isolate KPI violation root cause as described below. method to isolate the root cause of the KPI violation, as described
below.
Note: Whilst detection mode is a valuable tool for latency actions, Note: Whilst detection mode is a valuable tool for latency actions,
the authors feel that it is not justified to build the logic into the the authors feel that building the logic into the KPI system for QoS
KPI system for QoS configuration. As QoS-stamping is done configuration is not justified. As QoS stamping is done infrequently
infrequently and on a tiny percentage of user plane, it is more and on a tiny percentage of the user plane, it is more practical to
practical to use extended mode only for service chain QoS use extended mode only for service chain QoS verification.
verification.
5. Hybrid Models 5. Hybrid Models
A hybrid chain may be defined as a chain whereby there is a mix of A hybrid chain may be defined as a chain whereby there is a mix of
NSH-aware and NSH-unaware SFs. NSH-aware and NSH-unaware SFs.
Example 1. PNF in the middle Figure 12 shows an example of a hybrid chain with a PNF in the
middle.
Stamp Stamping
Controller Controller
| KPIDB | KPIDB
| SCP Interface | | SCP Interface |
,---. ,---. ,---. ,---. ,---. ,---. ,---. ,---.
/ \ / \ / \ / \ / \ / \ / \ / \
( SCL )-------->( SF1 )--------->( SF2 )--------->( SFn ) ( SCL )-------->( SF1 )--------->( SF2 )--------->( SFn )
\ FSN / \ / \ PNF1/ \ LSN / \ FSN / \ / \ PNF1/ \ LSN /
`---' `---' `---' `---' `---' `---' `---' `---'
Figure 12: Hybrid chain with PNF in middle
In this example the FSN begins operation and sets the SI to 3, SF1 Figure 12: Hybrid Chain with PNF in Middle
decrements this to 2 and passes the packet to an SFC proxy (not
shown).
The SFC proxy strips the NSH and passes to the PNF. On receipt back In this example, the FSN begins its operation and sets the SI to 3.
from the PNF, the proxy decrements the SI and passes the packet onto SF1 decrements the SI to 2 and passes the packet to an SFC proxy
the LSN with a SI=1. (not shown).
After the LSN processes the traffic it knows it is the last node on The SFC proxy strips the NSH and passes the packet to the PNF. On
the chain from the SI value and exports the entire NSH and all receipt back from the PNF, the proxy decrements the SI and passes the
metadata to the KPIDB. The payload is forwarded to the next hop on packet to the LSN with SI=1.
the underlay minus the NSH. The stamping information packet may be
given a new SPI to act as a homing tag to transport the stamp data
back to the KPIDB.
Example 2. PNF at the end After the LSN processes the traffic, it knows from the SI value that
it is the last node in the chain, and it exports the entire NSH and
all MD to the KPIDB. The payload is forwarded to the next hop on the
underlay minus the NSH. The stamping information packet may be given
a new SPI to act as a homing tag to transport the stamp data back to
the KPIDB.
Stamp Figure 13 shows an example of a hybrid chain with a PNF at the end.
Controller
| KPIDB
| SCP Interface |
,---. ,---. ,---. ,---.
/ \ / \ / \ / \
( SCL )-------->( SF1 )--------->( SF2 )--------->( PNFN )
\ FSN / \ / \ LSN / \ /
`---' `---' `---' `---'
Figure 13: Hybrid Chain with PNF at end
In this example the FSN begins operation and sets the SI to 3, the Stamping
SSI field set to 0x1, and the type to 1. Thus, when SF2 receives the Controller
packet with SI=1, it understands that it is expected to take on the | KPIDB
role of the LSN as it is the last NSH-aware node in the chain. | SCP Interface |
,---. ,---. ,---. ,---.
/ \ / \ / \ / \
( SCL )-------->( SF1 )--------->( SF2 )--------->( PNFN )
\ FSN / \ / \ LSN / \ /
`---' `---' `---' `---'
5.1. Targeted VNF Stamp Figure 13: Hybrid Chain with PNF at End
For the majority of flows within the service chain, stamps (ingress, In this example, the FSN begins its operation and sets the SI to 3.
egress or both) will be carried out at each hop until the SI The SSI field is set to 0x1, and the type is set to 1. Thus, when
decrements to zero and the NSH and Stamp MD is exported to the KPIDB. SF2 receives the packet with SI=1, it understands that it is expected
There may exist however the need to just test a particular VNF to take on the role of the LSN, as it is the last NSH-aware node in
(perhaps after a scale out operation, software upgrade or underlay the chain.
change for example). In this case the FSN should mark the NSH as
5.1. Targeted VNF Stamping
For the majority of flows within the service chain, stamps (Ingress
stamps, Egress stamps, or both) will be carried out at each hop until
the SI decrements to zero and the NSH and stamp MD are exported to
the KPIDB. However, the need to just test a particular VNF may exist
(perhaps after a scale-out operation, software upgrade, or underlay
change, for example). In this case, the FSN should mark the NSH as
follows: follows:
SSI field is set to 0x2. Type is set to the expected SI at the SF in o The SSI field is set to 0x2.
question. When outer SI is equal to the SSI, stamps are applied at SF
ingress and egress, and the NSH and MD are exported to the KPIDB.
6. Fragmentation Considerations o Type is set to the expected SI at the SF in question.
The method described in this document does not support fragmentation. o When the outer SI is equal to the SSI, stamps are applied at the
SF ingress and egress, and the NSH and MD are exported to the
KPIDB.
6. Fragmentation Considerations
The methods described in this document do not support fragmentation.
The SC should return an error should a stamping request from an The SC should return an error should a stamping request from an
external system exceed MTU limits and require fragmentation. external system exceed MTU limits and require fragmentation.
Depending on the length of the payload and the type of KPI-stamp and Depending on the length of the payload and the type of KPI stamp and
chain length, this will vary for each packet. chain length, this will vary for each packet.
In most service provider architectures we would expect a SI << 10, In most service provider architectures, we would expect SI << 10,
and that may include some PNFs in the chain which do not add which may include some PNFs in the chain that do not add overhead.
overhead. Thus for typical IMIX packet sizes we expect to able to Thus, for typical Internet Mix (IMIX) packet sizes [RFC6985], we
perform timestamping on the vast majority of flows without expect to be able to perform timestamping on the vast majority of
fragmenting. Thus the classifier can have a simple rule to only allow flows without fragmentation. Thus, the classifier can apply a simple
KPI-stamping on packet sizes less than 1200 bytes for example. rule that only allows KPI stamping on packet sizes less than 1200
bytes, for example.
7. Security Considerations 7. Security Considerations
The security considerations of NSH in general are discussed in The security considerations for the NSH in general are discussed in
[RFC8300]. [RFC8300].
The use of in-band timestamping, as defined in this document, can be In-band timestamping, as defined in this document, can be used as a
used as a means for network reconnaissance. By passively means for network reconnaissance. By passively eavesdropping on
eavesdropping to timestamped traffic, an attacker can gather timestamped traffic, an attacker can gather information about network
information about network delays and performance bottlenecks. delays and performance bottlenecks.
The NSH timestamp is intended to be used by various applications to The NSH timestamp is intended to be used by various applications to
monitor the network performance and to detect anomalies. Thus, a man- monitor network performance and to detect anomalies. Thus, a
in-the-middle attacker can maliciously modify timestamps in order to man-in-the-middle attacker can maliciously modify timestamps in order
attack applications that use the timestamp values. For example, an to attack applications that use the timestamp values. For example,
attacker could manipulate the SFC classifier operation, such that it an attacker could manipulate the SFC classifier operation, such that
forwards traffic through 'better' behaving chains. Furthermore, if it forwards traffic through "better-behaved" chains. Furthermore, if
timestamping is performed on a fraction of the traffic, an attacker timestamping is performed on a fraction of the traffic, an attacker
can selectively induce synthetic delay only to timestamped packets, can selectively induce synthetic delay only to timestamped packets
causing systematic error in the measurements. and can systematically trigger measurement errors.
Similarly, if an attacker can modify QoS stamps, erroneous values may Similarly, if an attacker can modify QoS stamps, erroneous values may
be imported into the KPIDB, resulting is further misconfiguration and be imported into the KPIDB, resulting in further misconfiguration and
subscriber QoE impairment. subscriber QoE impairment.
An attacker that gains access to the SCP can enable time and QoS- An attacker that gains access to the SCP can enable timestamping and
stamping for all subscriber flows, thereby causing performance QoS stamping for all subscriber flows, thereby causing performance
bottlenecks, fragmentation, or outages. bottlenecks, fragmentation, or outages.
As discussed in previous sections, NSH timestamping relies on an As discussed in previous sections, NSH timestamping relies on an
underlying time synchronization protocol. Thus, by attacking the time underlying time synchronization protocol. Thus, by attacking the
protocol an attack can potentially compromise the integrity of the time protocol, an attacker can potentially compromise the integrity
NSH timestamp. A detailed discussion about the threats against time of the NSH timestamp. A detailed discussion about the threats
protocols and how to mitigate them is presented in [RFC7384]. against time protocols and how to mitigate them is presented in
[RFC7384].
8. IANA Considerations
This document makes no requests for IANA action,
9. Contributors
This document originated as draft-browne-sfc-nsh-timestamp-00 and had
the following co-authors and contributors. We would like to thank and
recognize them and their contributions.
Yoram Moses
Technion
moses@ee.technion.ac.il
Brendan Ryan 8. IANA Considerations
Intel Corporation This document has no IANA actions.
brendan.ryan@intel.com 9. References
10. Acknowledgments 9.1. Normative References
This document was prepared using 2-Word-v2.0.template.dot. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
The authors gratefully acknowledge Mohamed Boucadair, Martin [RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function
Vigoureux and Adrian Farrel for their thorough reviews and helpful Chaining (SFC) Architecture", RFC 7665,
comments. DOI 10.17487/RFC7665, October 2015,
<https://www.rfc-editor.org/info/rfc7665>.
11. References [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in
RFC 2119 Key Words", BCP 14, RFC 8174,
DOI 10.17487/RFC8174, May 2017,
<https://www.rfc-editor.org/info/rfc8174>.
11.1. Normative References [RFC8300] Quinn, P., Ed., Elzur, U., Ed., and C. Pignataro, Ed.,
"Network Service Header (NSH)", RFC 8300,
DOI 10.17487/RFC8300, January 2018,
<https://www.rfc-editor.org/info/rfc8300>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 9.2. Informative References
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC7665] Halpern, J., Ed., and C. Pignataro, Ed., "Service [IEEE1588]
Function Chaining (SFC) Architecture", RFC 7665, DOI IEEE, "IEEE Standard for a Precision Clock Synchronization
10.17487/RFC7665, October 2015, <https://www.rfc- Protocol for Networked Measurement and Control Systems",
editor.org/info/rfc7665>. IEEE Standard 1588,
<https://standards.ieee.org/standard/1588-2008.html>.
[RFC8174] B. Leiba, "Ambiguity of Uppercase vs Lowercase in RFC [RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
2119 Key Words", RFC8174, May 2017. "Network Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010,
<https://www.rfc-editor.org/info/rfc5905>.
[RFC8300] Quinn, P., Elzur, U., Pignataro, C., "Network Service [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in
Header (NSH)", RFC 8300, 2018. Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384,
October 2014, <https://www.rfc-editor.org/info/rfc7384>.
11.2. Informative References [RFC6985] Morton, A., "IMIX Genome: Specification of Variable Packet
Sizes for Additional Testing", RFC 6985,
DOI 10.17487/RFC6985, July 2013,
<https://www.rfc-editor.org/info/rfc6985>.
[IEEE1588] IEEE TC 9 Instrumentation and Measurement Society, [Y.1731] ITU-T Recommendation G.8013/Y.1731, "Operations,
"1588 IEEE Standard for a Precision Clock administration and maintenance (OAM) functions and
Synchronization Protocol for Networked Measurement and mechanisms for Ethernet-based networks", August 2015,
Control Systems Version 2", IEEE Standard, 2008. <https://www.itu.int/rec/T-REC-G.8013/en>.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing [G.8261] ITU-T Recommendation G.8261/Y.1361, "Timing and
an IANA Considerations Section in RFCs", BCP 26, RFC synchronization aspects in packet networks", August 2013,
5226, May 2008. <https://www.itu.int/rec/T-REC-G.8261>.
[RFC5905] Mills, D., Martin, J., Burbank, J., Kasch, W., [G.8262] ITU-T Recommendation G.8262/Y.1362, "Timing
"Network Time Protocol Version 4: Protocol and characteristics of a synchronous Ethernet equipment slave
Algorithms Specification", RFC 5905, June 2010. clock", November 2018,
<https://www.itu.int/rec/T-REC-G.8262>.
[RFC7384] Mizrahi, T., "Security Requirements of Time Protocols [G.8264] ITU-T Recommendation G.8264/Y.1364, "Distribution of
in Packet Switched Networks", RFC 7384, October 2014. timing information through packet networks", August 2017,
<https://www.itu.int/rec/T-REC-G.8264>.
[TS] Mizrahi, T., Fabini, J., and A. Morton, "Guidelines [In-Situ-OAM]
for Defining Packet Timestamps", draft-ietf-ntp- Brockners, F., Bhandari, S., Pignataro, C., Gredler, H.,
packet-timestamps (work in progress), 2018. Leddy, J., Youell, S., Mizrahi, T., Mozes, D., Lapukhov,
P., Chang, R., Bernier, D., and J. Lemon, "Data Fields for
In-situ OAM", Work in Progress,
draft-ietf-ippm-ioam-data-05, March 2019.
[Y.1731] ITU-T Recommendation G.8013/Y.1731, "OAM Functions and Acknowledgments
Mechanisms for Ethernet-based Networks", August 2015.
[Y.1564] ITU-T Recommendation Y.1564, "Ethernet service The authors gratefully acknowledge Mohamed Boucadair, Martin
activation test methodology", March 2011. Vigoureux, and Adrian Farrel for their thorough reviews and helpful
comments.
[G.8261] ITU-T Recommendation G.8261/Y.1361, "Timing and Contributors
synchronization aspects in packet networks", August
2013.
[G.8262] ITU-T Recommendation G.8262/Y.1362, "Timing This document originated as draft-browne-sfc-nsh-timestamp-00; the
characteristics of a synchronous Ethernet equipment following people were coauthors of that draft. We would like to
slave clock", January 2015. thank them and recognize them for their contributions.
[G.8264] ITU-T Recommendation G.8264/Y.1364, "Distribution of Yoram Moses
timing information through packet networks", May 2014. Technion
Email: moses@ee.technion.ac.il
[I-D.ippm.ioam] Brendan Ryan
Brockners, Bhandari et al. "Data Fields for In-situ OAM" Intel Corporation
draft-ietf-ippm-ioam-data-03 (work in progress), June Email: brendan.ryan@intel.com
2018
Authors' Addresses Authors' Addresses
Rory Browne Rory Browne
Intel Intel
Dromore House Dromore House
Shannon Shannon
Co.Clare Co. Clare
Ireland Ireland
Email: rory.browne@intel.com Email: rorybrowne@yahoo.com
Andrey Chilikin Andrey Chilikin
Intel Intel
Dromore House Dromore House
Shannon Shannon
Co.Clare Co. Clare
Ireland Ireland
Email: andrey.chilikin@intel.com Email: andrey.chilikin@intel.com
Tal Mizrahi Tal Mizrahi
Huawei Network.IO Innovation Lab Huawei Network.IO Innovation Lab
Israel Israel
Email: tal.mizrahi.phd@gmail.com Email: tal.mizrahi.phd@gmail.com
 End of changes. 259 change blocks. 
662 lines changed or deleted 675 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/