RAW P. Thubert, Ed.
Internet-Draft Cisco Systems
Intended status: Informational D. Cavalcanti
Expires: January 2, 2020 Intel
X. Vilajosana
Universitat Oberta de Catalunya
C. Schmitt
Universität der Bundeswehr München
July 1, 2019

Reliable and Available Wireless Technologies


This document presents a series of recent technologies that are capable of time synchronization and scheduling of transmission, making them suitable to carry time-sensitive flows with requirements of both reliable delivery in bounded time, and availability at all times, regardless of packet transmission or individual equipement failures.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on January 2, 2020.

Copyright Notice

Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents

1. Introduction

When used in math or philosophy, the term "deterministic" generally refers to a perfection where all aspect are understood and predictable. A perfectly Deterministic Network would ensure that every packet reach its destination following a predetermined path along a predefined schedule to be delivered at the exact due time. In a real and imperfect world, a Deterministic Network must highly predictable, which is a combination of reliability and availability. On the one hand the network must be reliable, meaning that it will perform as expected for all packets and in particular that it will always deliver the packet at the destination in due time. On the other hand, the network must be available, meaning that it is resilient to any single outage, whether the cause is a software, a hardware or a transmission issue.

RAW (Reliable and Available Wireless) is an effort to provide Deterministic Networking on across a path that include a wireless physical layer. Making Wireless Reliable and Available is even more challenging than it is with wires, due to the numerous causes of loss in transmission that add up to the congestion losses and the delays caused by overbooked shared resources. In order to maintain a similar quality of service along a multihop path that is composed of wired and wireless hops, additional methods that are specific to wireless must be leveraged to combat the sources of loss that are also specific to wireless.

Such wireless-specific methods include per-hop retransmissions (HARQ) and P2MP overhearing whereby multiple receivers are scheduled to receive the same transmission, which balances the adverse effects of the transmission losses that are experienced when a radio is used as pure P2P.

2. Terminology

This specification uses several terms that are uncommon on protocols that ensure bets effort transmissions for stochastics flows, such as found in the traditional Internet and other statistically multiplexed packet networks.

Automatic Repeat Request, enabling an acknowledged transmission and retries. ARQ is a typical model at Layer-2 on a wireless medium. It is typically avoided end-to-end on deterministic flows because it introduces excessive indetermination in latency, but a limited number of retries within a bounded time may be used over a wireless link and yet respect end-to-end constraints.
That is exempt of unscheduled outage, the expectation for a network being that the flow is maintained in the face of any single breakage.
Forward error correction, sending redundant coded data to help the receiver recover transmission errors without the delays incurred with ARQ.
Hybrid ARQ, a combination of FEC and ARQ.
Path Computation Element.
PAREO (functions):
the wireless extension of DetNet PREOF. PAREO functions include scheduled ARQ at selected hops, and expect the use of new operations like overhearing where available.
That consistently performs as expected, the expectation for a network being to always deliver a packet in due time.
A DODAG oriented to a destination, and that enables Packet ARQ, Replication, Elimination, and Ordering Functions.

3. On Scheduling

The operations of a Deterministic Network often rely on precisely applying a tight schedule, in order to avoid collision loss and guarantee the worst-case time of delivery. To achieve this, there must be a shared sense of time throughout the network. The sense of time is usually provided by the lower layer and is not in scope for RAW.

3.1. Benefits of Scheduling on Wires

A network is reliable when the statistical effects that affect the packet transmission are eliminated. This involves maintaining at all time the amount of critical packets within the physical capabilities of the hardware and that of the radio medium. This is achieved by controlling the use of time-shared resources such as CPUs and buffers, by shaping the flows and by scheduling the time of transmission of the packets that compose the flow at every hop.

Equipment failure, such as an access point rebooting, a broken radio adapter, or a permanent obstacle to the transmission, is a secondary source of packet loss. When a breakage occurs, multiple packets are lost in a row before the flows are rerouted or the system may recover. This is not acceptable for critical applications such as related to safety. A typical process control loop will tolerate an occasional packet loss, but a loss of several packets in a row will cause an emergency stop (e.g., after 4 packets lost, within a period of 1 second).

Network Availability is obtained by making the transmission resilient against hardware failures and radio transmission losses due to uncontrolled events such as co-channel interferers, multipath fading or moving obstacles. The best results are typically achieved by pseudo randomly cumulating all forms of diversity, in the spatial domain with replication and elimination, in the time domain with ARQ and diverse scheduled transmissions, and in the frequency domain with frequency hopping or channel hopping between frames.

3.2. Benefits of Scheduling on Wireless

In addition to the benefits listed in Section 3.1, scheduling transmissions provides specific value to the wireless medium.

On the one hand, scheduling avoids collisions between scheduled transmissions and can ensure both time and frequency diversity between retries in order to defeat co-channel interference from un-controlled transmitters as well as multipath fading. Transmissions can be scheduled on multiple channels in parallel, which enables to use the full available spectrum while avoiding the hidden terminal problem, e.g., when the next packet in a same flow interferes on a same channel with the previous one that progressed a few hops farther.

On the other hand, scheduling optimizes the bandwidth usage: compared to classical Collision Avoidance techniques, there is no blank time related to inter-frame space (IFS) and exponential back-off in scheduled operations. A minimal Clear Channel Assessment may be needed to comply with the local regulations such as ETSI 300-328, but that will not detect a collision when the senders are synchronized. And because scheduling allows a time-sharing operation, there is no limit to the ratio of isolated critical traffic.

Finally, scheduling plays a critical role to save energy. In IOT, energy is the foremost concern, and synchronizing sender and listener enables to always maintain them in deep sleep when there is no scheduled transmission. This avoids idle listening and long preambles and enables long sleep periods between traffic and resynchronization, allowing battery-operated nodes to operate in a mesh topology for multiple years.

4. IEEE 802.11

4.1. Provenance and Documents

With an active portfolio of nearly 1,300 standards and projects under development, IEEE is a leading developer of industry standards in a broad range of technologies that drive the functionality, capabilities, and interoperability of products and services, transforming how people live, work, and communicate.

The IEEE 802 LAN/MAN Standards Committee (SC) develops and maintains networking standards and recommended practices for local, metropolitan, and other area networks, using an open and accredited process, and advocates them on a global basis. The most widely used standards are for Ethernet, Bridging and Virtual Bridged LANs Wireless LAN, Wireless PAN, Wireless MAN, Wireless Coexistence, Media Independent Handover Services, and Wireless RAN. An individual Working Group provides the focus for each area. Standards produced by the IEEE 802 SC are freely available from the IEEE GET Program after they have been published in PDF for six months.

The IEEE 802.11 LAN standards define the underlying MAC and PHY layers for the Wi-Fi technology. Wi-Fi/802.11 is one of the most successful wireless technologies, supporting many application domains. While previous 802.11 generations, such as 802.11n and 802.11ac, have focused mainly on improving peak throughput, more recent generations are also considering other performance vectors, such as efficiency enhancements for dense environments in 802.11ax, and latency and support for Time-Sensitive Networking (TSN) capabilities in 802.11be.

IEEE 802.11 already supports some 802.1 TSN standards and it is undergoing efforts to support for other 802.1 TSN capabilities required to address the use cases that require time synchronization and timeliness (bounded latency) guarantees with high reliability and availability. The IEEE 802.11 working group has been working in collaboration with the IEEE 802.1 group for several years extending 802.1 features over 802.11. As with any wireless media, 802.11 imposes new constraints and restrictions to TSN-grade QoS, and tradeoffs between latency and reliability guarantees must be considered as well as managed deployment requirements. An overview of 802.1 TSN capabilities and their extensions to 802.11 are discussed in [Cavalcanti_2019].

Wi-Fi Alliance (WFA) is the worldwide network of companies that drives global Wi-Fi adoption and evolution through thought leadership, spectrum advocacy, and industry-wide collaboration. The WFA work helps ensure that Wi-Fi devices and networks provide users the interoperability, security, and reliability they have come to expect.

The following IEEE 802.11 specifications/certifications are relevant in the context of reliable and available wireless services and support for time-sensitive networking capabilities:

Time Synchronization:
IEEE802.11-2016 with IEEE802.1AS; WFA TimeSync Certification.
Congestion Control:
IEEE802.11-2016 Admission Control; WFA Admission Control.
WFA Wi-Fi Protected Access, WPA2 and WPA3.
Interoperating with IEEE802.1Q bridges:
Stream Reservation Protocol (part of IEEE802.1Qat):
Scheduled channel access:
IEEE802.11ad Enhancements for very high throughput in the 60 GHz band [IEEE80211ad].
802.11 Real-Time Applications:
Topic Interest Group (TIG) ReportDoc [IEEE_doc_11-18-2009-06].

In addition, major amendments being developed by the IEEE802.11 Working Group include capabilities that can be used as the basis for providing more reliable and predictable wireless connectivity and support time-sensitive applications:

IEEE 802.11ax D4.0: Enhancements for High Efficiency (HE).
IEEE 802.11be Extreme High Throughput (EHT).
IEE 802.11ay Enhanced throughput for operation in license-exempt bands above 45 GHz.

The main 802.11ax and 802.11be capabilities and their relevance to RAW are discussed in the remainder of this document.

4.2. 802.11ax High Efficiency (HE)

4.2.1. General Characteristics

The next generation Wi-Fi (Wi-Fi 6) is based on the IEEE802.11ax amendment [IEEE80211ax], which includes new capabilities to increase efficiency, control and reduce latency. Some of the new features include higher order 1024-QAM modulation, support for uplink multi-user MIMO, OFDMA, trigger-based access and Target Wake time (TWT) for enhanced power savings. The OFDMA mode and trigger-based access enable scheduled operation, which is a key capability required to support deterministic latency and reliability for time-sensitive flows. 802.11ax can operate in up to 160 MHz channels and it includes support for operation in the new 6 GHz band, which is expected to be open to unlicensed use by the FCC and other regulatory agencies worldwide. Multi-User OFDMA and Trigger-based Scheduled Access

802.11ax introduced a new orthogonal frequency-division multiple access (OFDMA) mode in which multiple users can be scheduled across the frequency domain. In this mode, the Access Point (AP) can initiate multi-user (MU) Uplink (UL) transmissions in the same PHY Protocol Data Unit (PPDU) by sending a trigger frame. This centralized scheduling capability gives the AP much more control of the channel, and it can remove contention between devices for uplink transmissions, therefore reducing the randomness caused by CSMA-based access between stations. The AP can also transmit simultaneously to multiple users in the downlink direction by using a Downlink (DL) MU OFDMA PPDU. In order to initiate a contention free Transmission Opportunity (TXOP) using the OFDMA mode, the AP still follows the typical listen before talk procedure to acquire the medium, which ensures interoperability and compliance with unlicensed band access rules. However, 802.11ax also includes a multi-user Enhanced Distributed Channel Access (MU-EDCA) capability, which allows the AP to get higher channel access priority. Improved PHY Robustness

The 802.11ax PHY can operate with 0.8, 1.6 or 3.2 microsecond guard interval (GI). The larger GI options provide better protection against multipath, which is expected to be a challenge in industrial environments. The possibility to operate with smaller resource units (e.g. 2 MHz) enabled by OFDMA also helps reduce noise power and improve SNR, leading to better packet error rate (PER) performance.

802.11ax supports beamforming as in 802.11ac, but introduces UL MU MIMO, which helps improve reliability. The UL MU MIMO capability is also enabled by the trigger based access operation in 802.11ax. Support for 6GHz band

The 802.11ax specification [IEEE80211ax] includes support for operation in the new 6 GHz band. Given the amount of new spectrum available as well as the fact that no legacy 802.11 device (prior 802.11ax) will be able to operate in this new band, 802.11ax operation in this new band can be even more efficient.

4.2.2. Applicability to deterministic flows

TSN capabilities, as defined by the IEEE 802.1 TSN standards, provide the underlying mechanism for supporting deterministic flows in a Local Area Network (LAN). The 802.11 working group has already incorporated support for several TSN capabilities, so that time-sensitive flow can experience precise time synchronization and timeliness when operating over 802.11 links. TSN capabilities supported over 802.11 (which also extends to 802.11ax), include:

802.1AS based Time Synchronization (other time synchronization techniques may also be used)
Interoperating with IEEE802.1Q bridges
Time-sensitive Traffic Stream identification

The exiting 802.11 TSN capabilities listed above, and the 802.11ax OFDMA and scheduled access provide a new set of tools to better server time-sensitive flows. However, it is important to understand the tradeoffs and constraints associated with such capabilities, as well as redundancy and diversity mechanisms that can be used to provide more predictable and reliable performance. 802.11 Managed network operation and admission control

Time-sensitive applications and TSN standards are expected to operate under a managed network (e.g. industrial/enterprise network). Thus, the Wi-Fi operation must also be carefully managed and integrated with the overall TSN management framework, as defined in the IEEE Std. 802.1Qcc specification [IEEE8021Qcc].

Some of the random-access latency and interference from legacy/unmanaged devices can be minimized under a centralized management mode as defined in IEEE Std. 802.1Qcc, in which admission control procedures are enforced.

Existing traffic stream identification, configuration and admission control procedures defined in IEEE Std. 802.11 QoS mechanism can be re-used. However, given the high degree of determinism required by many time-sensitive applications, additional capabilities to manage interference and legacy devices within tight time-constraints need to be explored. Scheduling for bounded latency and diversity

As discussed earlier, the 802.11ax OFDMA mode introduces the possibility of assigning different RUs (frequency resources) to users within a PPDU. Several RU sizes are defined in the specification (26, 52, 106, 242, 484, 996 subcarriers). In addition, the AP can also decide on MCS and grouping of users within a given OFMDA PPDU. Such flexibility can be leveraged to support time-sensitive applications with bounded latency, especially in a managed network where stations can be configured to operate under the control of the AP.

As shown in [Cavalcanti_2019], it is possible to achieve latencies in the order of 1msec with high reliability in an interference free environment. Obviously, there are latency, reliability and capacity tradeoffs to be considered. For instance, smaller Resource Units (RU)s result in longer transmission durations, which may impact the minimal latency that can be achieved, but the contention latency and randomness elimination due to multi-user transmission is a major benefit of the OFDMA mode.

The flexibility to dynamically assign RUs to each transmission also enables the AP to provide frequency diversity, which can help increase reliability.

4.3. 802.11be Extreme High Throughput (EHT)

4.3.1. General Characteristics

The 802.11be is the next major 802.11 amendment (after 802.11ax) for operation in the 2.4, 5 and 6 GHz bands. 802.11be is expected to include new PHY and MAC features and it is targeting extremely high throughput (at least 30 Gbps), as well as enhancements to worst case latency and jitter. It is also expected to improve the integration with 802.1 TSN to support time-sensitive applications over Ethernet and Wireless LANs.

The 802.11be Task Group started its operation in May 2019, therefore, detailed information about specific features is not yet available. Only high level candidate features have been discussed so far, including:

320MHz bandwidth and more efficient utilization of non-contiguous spectrum.
Multi-band/multi-channel aggregation and operation.
16 spatial streams and related MIMO enhancements.
Multi-Access Point (AP) Coordination.
Enhanced link adaptation and retransmission protocol, e.g. Hybrid Automatic Repeat Request (HARQ).
Any required adaptations to regulatory rules for the 6 GHz spectrum.

4.3.2. Applicability to deterministic flows

The 802.11 Real-Time Applications (RTA) Topic Interest Group (TIG) provided detailed information on use cases, issues and potential solution directions to improve support for time-sensitive applications in 802.11. The RTA TIG report [IEEE_doc_11-18-2009-06] was used as input to the 802.11be project scope.

Improvements for worst-case latency, jitter and reliability were the main topics identified in the RTA report, which were motivated by applications in gaming, industrial automation, robotics, etc. The RTA report also highlighted the need to support additional TSN capabilities, such as time-aware (802.1Qbv) shaping and packet replication and elimination as defined in 802.1CB.

802.11be is expected to build on and enhance 802.11ax capabilities to improve worst case latency and jitter. Some of the enhancement areas are discussed next. Enhanced scheduled operation for bounded latency

In addition to the throughput enhancements, 802.11be will leverage the trigger-based scheduled operation enabled by 802.11ax to provide efficient and more predictable medium access. 802.11be is expected to include enhancements to reduce overhead and enable more efficient operation in managed network deployments [IEEE_doc_11-19-0373-00]. Multi-AP coordination

Multi-AP coordination is one of the main new candidate features in 802.11be. It can provide benefits in throughput and capacity and has the potential to address some of the issues that impact worst case latency and reliability. Multi-AP coordination is expected to address the contention due to overlapping Basic Service Sets (OBSS), which is one of the main sources of random latency variations. 802.11be can define methods to enable better coordination between APs, for instance, in a managed network scenario, in order to reduce latency due to unmanaged contention.

Several multi-AP coordination approaches have been discussed with different levels of complexities and benefits, but specific coordination methods have not yet been defined. Multi-band operation

802.11be will introduce new features to improve operation over multiple bands and channels. By leveraging multiple bands/channels, 802.11be can isolate time-sensitive traffic from network congestion, one of the main causes of large latency variations. In a managed 802.11be network, it should be possible to steer traffic to certain bands/channels to isolate time-sensitive traffic from other traffic and help achieve bounded latency.

4.4. 802.11ad and 802.11ay (mmWave operation)

4.4.1. General Characteristics

The IEEE 802.11ad amendment defines PHY and MAC capabilities to enable multi-Gbps throughput in the 60 GHz millimeter wave (mmWave) band. The standard addresses the adverse mmWave signal propagation characteristics and provides directional communication capabilities that take advantage of beamforming to cope with increased attenuation. An overview of the 802.11ad standard can be found in [Nitsche_2015] .

The IEEE 802.11ay is currently developing enhancements to the 802.11ad standard to enable the next generation mmWave operation targeting 100 Gbps throughput. Some of the main enhancements in 802.11ay include MIMO, channel bonding, improved channel access and beamforming training. An overview of the 802.11ay capabilities can be found in [Ghasempour_2017]

4.4.2. Applicability to deterministic flows

The high data rates achievable with 802.11ad and 802.11ay can significantly reduce latency down to microsecond levels. Limited interference from legacy and other unlicensed devices in 60 GHz is also a benefit. However, directionality and short range typical in mmWave operation impose new challenges such as the overhead required for beam training and blockage issues, which impact both latency and reliability. Therefore, it is important to understand the use case and deployment conditions in order to properly apply and configure 802.11ad/ay networks for time sensitive applications.

The 802.11ad standard include a scheduled access mode in which stations can be allocated contention-free service periods by a central controller. This scheduling capability is also available in 802.11ay, and it is one of the mechanisms that can be used to provide bounded latency to time-sensitive data flows. An analysis of the theoretical latency bounds that can be achieved with 802.11ad service periods is provided in [Cavalcanti_2019].

5. IEEE 802.15.4

5.1. Provenance and Documents

The IEEE802.15.4 Task Group has been driving the development of low-power low-cost radio technology. The IEEE802.15.4 physical layer has been designed to support demanding low-power scenarios targeting the use of unlicensed bands, both the 2.4 GHz and sub GHz Industrial, Scientific and Medical (ISM) bands. This has imposed requirements in terms of frame size, data rate and bandwidth to achieve reduced collision probability, reduced packet error rate, and acceptable range with limited transmission power. The PHY layer supports frames of up to 127 bytes. The Medium Access Control (MAC) sublayer overhead is in the order of 10-20 bytes, leaving about 100 bytes to the upper layers. IEEE802.15.4 uses spread spectrum modulation such as the Direct Sequence Spread Spectrum (DSSS).

The Timeslotted Channel Hopping (TSCH) mode was added to the 2015 revision of the IEEE802.15.4 standard [IEEE802154]. TSCH is targeted at the embedded and industrial world, where reliability, energy consumption and cost drive the application space.

At the IETF, the 6TiSCH Working Group (WG) [TiSCH] deals with best effort operation of IPv6 [RFC8200] over TSCH. 6TiSCH has enabled distributed scheduling to exploit the deterministic access capabilities provided by TSCH. The group designed the essential mechanisms to enable the management plane operation while ensuring IPv6 is supported. Yet the charter did not focus to providing a solution to establish end to end Tracks while meeting quality of service requirements. 6TiSCH, through the RFC8480 [RFC8480] defines the 6P protocol which provides a pairwise negotiation mechanism to the control plane operation. The protocol supports agreement on a schedule between neighbors, enabling distributed scheduling. 6P goes hand-in-hand with a Scheduling Function (SF), the policy that decides how to maintain cells and trigger 6P transactions. The Minimal Scheduling Function (MSF) [I-D.ietf-6tisch-msf] is the default SF defined by the 6TiSCH WG; other standardized SFs can be defined in the future. MSF extends the minimal schedule configuration, and is used to add child-parent links according to the traffic load.

Time sensitive networking on low power constrained wireless networks have been partially addressed by ISA100.11a [ISA100.11a] and WirelessHART [WirelessHART]. Both technologies involve a central controller that computes redundant paths for industrial process control traffic over a TSCH mesh. Moreover, ISA100.11a introduces IPv6 capabilities with a Link-Local Address for the join process and a global unicast addres for later exchanges, but the IPv6 traffic typically ends at a local application gateway and the full power of IPv6 for end-to-end communication is not enabled. Compared to that state of the art, work at the IETF and in particular at RAW could provide additional techniques such as optimized P2P routing, PAREO functions, and end-to-end secured IPv6/CoAP connectivity.

The 6TiSCH architecture [I-D.ietf-6tisch-architecture] identifies different models to schedule resources along so-called Tracks (see Section exploiting the TSCH schedule structure however the focus at 6TiSCH is on best effort traffic and the group was never chartered to produce standard work related to Tracks.

Useful References include:

IEEE Std 802.15.4: "IEEE Std. 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks" [IEEE802154]. The latest version at the time of this writing is dated year 2015.
Morell, A. , Vilajosana, X. , Vicario, J. L. and Watteyne, T. (2013), Label switching over IEEE802.15.4e networks. Trans. Emerging Tel. Tech., 24: 458-475. doi:10.1002/ett.2650" [morell13].
De Armas, J., Tuset, P., Chang, T., Adelantado, F., Watteyne, T., Vilajosana, X. (2016, September). Determinism through path diversity: Why packet replication makes sense. In 2016 International Conference on Intelligent Networking and Collaborative Systems (INCoS) (pp. 150-154). IEEE. [dearmas16].
X. Vilajosana, T. Watteyne, M. Vucinic, T. Chang and K. S. J. Pister, "6TiSCH: Industrial Performance for IPv6 Internet-of-Things Networks," in Proceedings of the IEEE, vol. 107, no. 6, pp. 1153-1165, June 2019. [vilajosana19].

5.2. TimeSlotted Channel Hopping

5.2.1. General Characteristics

As a core technique in IEEE802.15.4, TSCH splits time in multiple time slots that repeat over time. A set of timeslots constructs a Slotframe (see Section For each timeslot, a set of available frequencies can be used, resulting in a matrix-like schedule (see Figure 1).

                       timeslot offset
     | 0    1    2    3    4  | 0    1    2    3    4  |    Nodes
     +------------------------+------------------------+   +-----+
     |    |    |    |    |    |    |    |    |    |    |   |  C  |
CH-1 | EB |    |    |C->B|    | EB |    |    |C->B|    |   |     |
     |    |    |    |    |    |    |    |    |    |    |   +-----+
     +-------------------------------------------------+      |
     |    |    |    |    |    |    |    |    |    |    |      |
CH-2 |    |    |B->C|    |B->A|    |    |B->C|    |B->A|   +-----+
     |    |    |    |    |    |    |    |    |    |    |   |  B  |
     +-------------------------------------------------+   |     |
 ...                                                       +-----+  
     +-------------------------------------------------+      |
     |    |    |    |    |    |    |    |    |    |    |   +-----+     
CH-15|    |A->B|    |    |    |    |A->B|    |    |    |   |  A  |
     |    |    |    |    |    |    |    |    |    |    |   |     |
     +-------------------------------------------------+   +-----+ 

Figure 1: Slotframe example with scheduled cells between nodes A, B and C

This schedule represents the possible communications of a node with its neighbors, and is managed by a Scheduling Function such as the Minimal Scheduling Function (MSF) [I-D.ietf-6tisch-msf]. Each cell in the schedule is identified by its slotoffset and channeloffset coordinates. A cell's timeslot offset indicates its position in time, relative to the beginning of the slotframe. A cell's channel offset is an index which maps to a frequency at each iteration of the slotframe. Each packet exchanged between neighbors happens within one cell. The size of a cell is a timeslot duration, between 10 to 15 milliseconds. An Absolute Slot Number (ASN) indicates the number of slots elapsed since the network started. It increments at every slot. This is a 5 byte counter that can support networks running for more than 300 years without wrapping (assuming a 10 ms timeslot). Channel hopping provides increased reliability to multi-path fading and external interference. It is handled by TSCH through a channel hopping sequence referred as macHopSeq in the IEEE802.15.4 specification.

The Time-Frequency Division Multiple Access provided by TSCH enables the orchestration of traffic flows, spreading them in time and frequency, and hence enabling an efficient management of the bandwidth utilization. Such efficient bandwidth utilization can be combined to OFDM modulations also supported by the IEEE802.15.4 standard [IEEE802154] since the 2015 version.

In the RAW context, low power reliable networks should address non-critical control scenarios such as Class 2 and monitoring scenarios such as Class 4 defined by the RFC5673 [RFC5673]. As a low power technology targeting industrial scenarios radio transducers provide low data rates (typically between 50kbps to 250kbps) and robust modulations to trade-off performance to reliability. TSCH networks are organized in mesh topologies and connected to a backbone. Latency in the mesh network is mainly influenced by propagation aspects such as interference. ARQ methods and redundancy techniques such as replication and elimination should be studied to provide the needed performance to address deterministic scenarios.

5.2.2. Applicability to Deterministic Flows

Nodes in a TSCH network are tightly synchronized. This enables to build the slotted structure and ensure efficient utilization of resources thanks to proper scheduling policies. Scheduling is a key to orchestrate the resources that different nodes in a Track or a path are using. Slotframes can be split in resource blocks reserving the needed capacity to certain flows. Periodic and bursty traffic can be handled independently in the schedule, using active and reactive policies and taking advantage of overprovisionned cells to measu reth excursion. Along a Track, resource blocks can be chained so nodes in previous hops transmit their data before the next packet comes. This provides a tight control to latency along a Track. Collision loss is avoided for best effort traffic by overprovisionning resources, giving time to the management plane of the network to dedicate more resources if needed. Centralized Path Computation

In a controlled environment, a 6TiSCH device usually does not place a request for bandwidth between itself and another device in the network. Rather, an Operation Control System (OCS) invoked through an Human/Machine Interface (HMI) iprovides the Traffic Specification, in particular in terms of latency and reliability, and the end nodes, to a Path Computation element (PCE). With this, the PCE computes a Track between the end nodes and provisions every hop in the Track with per-flow state that describes the per-hop operation for a given packet, the corresponding timeSlots, and the flow identification to recognize which packet is placed in which Track, sort out duplicates, etc. In Figure 2, an example of Operational Control System and HMI is depicted.

For a static configuration that serves a certain purpose for a long period of time, it is expected that a node will be provisioned in one shot with a full schedule, which incorporates the aggregation of its behavior for multiple Tracks. The 6TiSCH Architecture expects that the programing of the schedule is done over CoAP as discussed in "6TiSCH Resource Management and Interaction using CoAP".

But an Hybrid mode may be required as well whereby a single Track is added, modified, or removed, for instance if it appears that a Track does not perform as expected for, say, Packet Delivery Ratio (PDR). For that case, the expectation is that a protocol that flows along a Track (to be), in a fashion similar to classical Traffic Engineering (TE) [CCAMP], may be used to update the state in the devices. 6TiSCH provides means for a device to negotiate a timeSlot with a neighbor, but in general that flow was not designed and no protocol was selected and it is expected that DetNet will determine the appropriate end-to-end protocols to be used in that case.

Stream Management Entity

                      Operational Control System and HMI
   -+-+-+-+-+-+-+ Northbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-

             PCE         PCE              PCE              PCE

   -+-+-+-+-+-+-+ Southbound -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-

           --- 6TiSCH------6TiSCH------6TiSCH------6TiSCH--
  6TiSCH /     Device      Device      Device      Device   \
  Device-                                                    - 6TiSCH
         \     6TiSCH      6TiSCH      6TiSCH      6TiSCH   /  Device

Figure 2 Packet Marking and Handling

Section "Packet Marking and Handling" of [I-D.ietf-6tisch-architecture] describes the packet tagging and marking that is expected in 6TiSCH networks. Tagging Packets for Flow Identification

For packets that are routed by a PCE along a Track, the tuple formed by the IPv6 source address and a local RPLInstanceID is tagged in the packets to identify uniquely the Track and associated transmit bundle of timeSlots.

It results that the tagging that is used for a DetNet flow outside the 6TiSCH LLN MUST be swapped into 6TiSCH formats and back as the packet enters and then leaves the 6TiSCH network.

Note: The method and format used for encoding the RPLInstanceID at 6lo is generalized to all 6TiSCH topological Instances, which includes Tracks. Replication, Retries and Elimination

PRE establishes several paths in a network to provide redundancy and parallel transmissions to bound the end-to-end delay. Considering the scenario shown in Figure 3, many different paths are possible for S to reach R. A simple way to benefit from this topology could be to use the two independent paths via nodes A, C, E and via B, D, F. But more complex paths are possible as well.

                 (A)   (C)   (E)   
   source (S)                       (R) (destination)
                 (B)   (D)   (F)  


Figure 3: A Typical Ladder Shape with Two Parallel Paths Toward the Destination

By employing a Packet Replication function, each node forwards a copy of each data packet over two different branches. For instance, in Figure 4, the source node S transmits the data packet to nodes A and B, in two different timeslots within the same TSCH slotframe.

               ===> (A) => (C) => (E) === 
             //        \\//   \\//       \\
   source (S)          //\\   //\\         (R) (destination)
             \\       //  \\ //  \\      //
               ===> (B) => (D) => (F) ===


Figure 4: Packet Replication: S transmits twice the same data packet, to its DP (A) and to its AP (B).

By employing Packet Elimination function once a node receives the first copy of a data packet, it discards the subsequent copies. Because the first copy that reaches a node is the one that matters, it is the only copy that will be forwarded upward.

Considering that the wireless medium is broadcast by nature, any neighbor of a transmitter may overhear a transmission. By employing the Promiscuous Overhearing function, nodes will have multiple opportunities to receive a given data packet. For instance, in Figure 4, when the source node S transmits the data packet to node A, node B may overhear this transmission.

6TiSCH expects elimination and replication of packets along a complex Track, but has no position about how the sequence numbers would be tagged in the packet.

As it goes, 6TiSCH expects that timeSlots corresponding to copies of a same packet along a Track are correlated by configuration, and does not need to process the sequence numbers.

The semantics of the configuration MUST enable correlated timeSlots to be grouped for transmit (and respectively receive) with a 'OR' relations, and then a 'AND' relation MUST be configurable between groups. The semantics is that if the transmit (and respectively receive) operation succeeded in one timeSlot in a 'OR' group, then all the other timeSLots in the group are ignored. Now, if there are at least two groups, the 'AND' relation between the groups indicates that one operation must succeed in each of the groups.

On the transmit side, timeSlots provisioned for retries along a same branch of a Track are placed a same 'OR' group. The 'OR' relation indicates that if a transmission is acknowledged, then further transmissions SHOULD NOT be attempted for timeSlots in that group. There are as many 'OR' groups as there are branches of the Track departing from this node. Different 'OR' groups are programmed for the purpose of replication, each group corresponding to one branch of the Track. The 'AND' relation between the groups indicates that transmission over any of branches MUST be attempted regardless of whether a transmission succeeded in another branch. It is also possible to place cells to different next-hop routers in a same 'OR' group. This allows to route along multi-path Tracks, trying one next-hop and then another only if sending to the first fails.

On the receive side, all timeSlots are programmed in a same 'OR' group. Retries of a same copy as well as converging branches for elimination are converged, meaning that the first successful reception is enough and that all the other timeSlots can be ignored. Differentiated Services Per-Hop-Behavior

Additionally, an IP packet that is sent along a Track uses the Differentiated Services Per-Hop-Behavior Group called Deterministic Forwarding, as described in [I-D.svshah-tsvwg-deterministic-forwarding]. Topology and capabilities

6TiSCH nodes are usually IoT devices, characterized by very limited amount of memory, just enough buffers to store one or a few IPv6 packets, and limited bandwidth between peers. It results that a node will maintain only a small number of peering information, and will not be able to store many packets waiting to be forwarded. Peers can be identified through MAC or IPv6 addresses.

Neighbors can be discovered over the radio using mechanism such as Enhanced Beacons, but, though the neighbor information is available in the 6TiSCH interface data model, 6TiSCH does not describe a protocol to pro-actively push the neighborhood information to a PCE. This protocol should be described and should operate over CoAP. The protocol should be able to carry multiple metrics, in particular the same metrics as used for RPL operations [RFC6551].

The energy that the device consumes in sleep, transmit and receive modes can be evaluated and reported. So can the amount of energy that is stored in the device and the power that it can be scavenged from the environment. The PCE SHOULD be able to compute Tracks that will implement policies on how the energy is consumed, for instance balance between nodes, ensure that the spent energy does not exceeded the scavenged energy over a period of time, etc... Schedule Management by a PCE

6TiSCH supports a mixed model of centralized routes and distributed routes. Centralized routes can for example be computed by a entity such as a PCE [PCE]. Distributed routes are computed by RPL.

Both methods may inject routes in the Routing Tables of the 6TiSCH routers. In either case, each route is associated with a 6TiSCH topology that can be a RPL Instance topology or a Track. The 6TiSCH topology is indexed by a Instance ID, in a format that reuses the RPLInstanceID as defined in RPL.

Both RPL and PCE rely on shared sources such as policies to define Global and Local RPLInstanceIDs that can be used by either method. It is possible for centralized and distributed routing to share a same topology. Generally they will operate in different slotFrames, and centralized routes will be used for scheduled traffic and will have precedence over distributed routes in case of conflict between the slotFrames. SlotFrames and Priorities

A slotFrame is the base object that a PCE needs to manipulate to program a schedule into an LLN node. Elaboration on that concept can be fond in section "SlotFrames and Priorities" of [I-D.ietf-6tisch-architecture]

IEEE802.15.4 TSCH avoids contention on the medium by formatting time and frequencies in cells of transmission of equal duration. In order to describe that formatting of time and frequencies, the 6TiSCH architecture defines a global concept that is called a Channel Distribution and Usage (CDU) matrix; a CDU matrix is a matrix of cells with an height equal to the number of available channels (indexed by ChannelOffsets) and a width (in timeSlots) that is the period of the network scheduling operation (indexed by slotOffsets) for that CDU matrix. The size of a cell is a timeSlot duration, and values of 10 to 15 milliseconds are typical in 802.15.4 TSCH to accommodate for the transmission of a frame and an acknowledgement, including the security validation on the receive side which may take up to a few milliseconds on some device architecture.

The frequency used by a cell in the matrix rotates in a pseudo-random fashion, from an initial position at an epoch time, as the matrix iterates over and over.

A CDU matrix is computed by the PCE, but unallocated timeSlots may be used opportunistically by the nodes for classical best effort IP traffic. The PCE has precedence in the allocation in case of a conflict.

In a given network, there might be multiple CDU matrices that operate with different width, so they have different durations and represent different periodic operations. It is recommended that all CDU matrices in a 6TiSCH domain operate with the same cell duration and are aligned, so as to reduce the chances of interferences from slotted-aloha operations. The PCE MUST compute the CDU matrices and shared that knowledge with all the nodes. The matrices are used in particular to define slotFrames.

A slotFrame is a MAC-level abstraction that is common to all nodes and contains a series of timeSlots of equal length and precedence. It is characterized by a slotFrame_ID, and a slotFrame_size. A slotFrame aligns to a CDU matrix for its parameters, such as number and duration of timeSlots.

Multiple slotFrames can coexist in a node schedule, i.e., a node can have multiple activities scheduled in different slotFrames, based on the precedence of the 6TiSCH topologies. The slotFrames may be aligned to different CDU matrices and thus have different width. There is typically one slotFrame for scheduled traffic that has the highest precedence and one or more slotFrame(s) for RPL traffic. The timeSlots in the slotFrame are indexed by the SlotOffset; the first cell is at SlotOffset 0.

The 6TiSCH architecture introduces the concept of chunks ([I-D.ietf-6tisch-architecture]) to operate such spectrum distribution for a whole group of cells at a time. The CDU matrix is formatted into a set of chunks, each of them identified uniquely by a chunk-ID, see Figure 5. The PCE MUST compute the partitioning of CDU matrices into chunks and shared that knowledge with all the nodes in a 6TiSCH network.

             +-----+-----+-----+-----+-----+-----+-----+     +-----+
chan.Off. 0  |chnkA|chnkP|chnk7|chnkO|chnk2|chnkK|chnk1| ... |chnkZ|
             +-----+-----+-----+-----+-----+-----+-----+     +-----+
chan.Off. 1  |chnkB|chnkQ|chnkA|chnkP|chnk3|chnkL|chnk2| ... |chnk1|
             +-----+-----+-----+-----+-----+-----+-----+     +-----+
             +-----+-----+-----+-----+-----+-----+-----+     +-----+
chan.Off. 15 |chnkO|chnk6|chnkN|chnk1|chnkJ|chnkZ|chnkI| ... |chnkG|
             +-----+-----+-----+-----+-----+-----+-----+     +-----+
                0     1     2     3     4     5     6          M

Figure 5: CDU matrix Partitioning in Chunks

The appropriation of a chunk can be requested explicitly by the PCE to any node. After a successful appropriation, the PCE owns the cells in that chunk, and may use them as hard cells to set up Tracks. Then again, 6TiSCH did not propose a method for chunk definition and a protocol for appropriation. This is to be done at RAW. 6TiSCH Tracks

A Track at 6TiSCH is the application to wireless of the concept of a path in the Detnet architecture [I-D.ietf-detnet-architecture]. A Track can follow a simple sequence of relay nodes or can be structured as a more complex Destination Oriented Directed Acyclic Graph (DODAG) to a unicast destination. Along a Track, 6TiSCH nodes reserve the resources to enable the efficient transmission of packets while aiming to optimize certain properties such as reliability and ensure small jitter or bounded latency. The Track structure enables Layer-2 forwarding schemes, reducing the overhead of taking routing decisions at the Layer-3.

Serial Tracks can be understood as the concatenation of cells or bundles along a routing path from a source towards a destination. The serial Track concept is analogous to the circuit concept where resources are chained through the multi-hop topology. For example, A bundle of Tx Cells in a particular node is paired to a bundle of Rx Cells in the next hop node following a routing path.

Whereas scheduling ensures reliable delivery in bounded time along any Track, high availability requires the application of PAREO functions along a more complex DODAG Track structure. A DODAG has forking and joining nodes where the concepts such as Replication and Elimination can be exploited. Spatial redundancy increases the oveall energy consumption in the network but improves significantly the availability of the network as well as the packet delivery ratio. A Track may also branch off and rejoin, for the purpose of the so-called Packet Replication and Elimination (PRE), over non congruent branches. PRE may be used to complement layer-2 Automatic Repeat reQuest (ARQ) and receiver-end Ordering to form the PAREO functions. PAREO functions enable to meet industrial expectations in PDR within bounded delivery time over a Track that includes wireless links, even when the Track extends beyond the 6TiSCH network.

                  | IoT |
                  | G/W |
                     ^  <---- Elimination
                    | |
     Track branch   | |    
            +-------+ +--------+ Subnet Backbone  
            |                  |   
         +--|--+            +--|--+ 
         |  |  | Backbone   |  |  | Backbone
    o    |  |  | router     |  |  | router  
         +--/--+            +--|--+         
    o     /    o     o---o----/       o   
        o    o---o--/   o      o   o  o   o    
   o     \  /     o               o   LLN    o      
      o   v  <---- Replication

Figure 6: End-to-End deterministic Track

In the example above (see Figure 6), a Track is laid out from a field device in a 6TiSCH network to an IoT gateway that is located on a IEEE802.1 TSN backbone.

The Replication function in the field device sends a copy of each packet over two different branches, and a PCE schedules each hop of both branches so that the two copies arrive in due time at the gateway. In case of a loss on one branch, hopefully the other copy of the packet still makes it in due time. If two copies make it to the IoT gateway, the Elimination function in the gateway ignores the extra packet and presents only one copy to upper layers.

At each 6TiSCH hop along the Track, the PCE may schedule more than one timeSlot for a packet, so as to support Layer-2 retries (ARQ). It is also possible that the field device only uses the second branch if sending over the first branch fails.

In current deployments, a TSCH Track does not necessarily support PRE but is systematically multi-path. This means that a Track is scheduled so as to ensure that each hop has at least two forwarding solutions, and the forwarding decision is to try the preferred one and use the other in case of Layer-2 transmission failure as detected by ARQ.

Methods to implement complex Tracks are described in [I-D.papadopoulos-paw-pre-reqs] and complemented by extensions to the RPL routing protocol in [I-D.ietf-roll-nsa-extension] for best effort traffic, but a centralized routing technique such as promoted in DetNet is still missing. Track Scheduling Protocol

Section "Schedule Management Mechanisms" of the 6TiSCH architecture describes 4 paradigms to manage the TSCH schedule of the LLN nodes: Static Scheduling, neighbor-to-neighbor Scheduling, remote monitoring and scheduling management, and Hop-by-hop scheduling. The Track operation for DetNet corresponds to a remote monitoring and scheduling management by a PCE.

Early work at 6TiSCH on a data model and a protocol to program the schedule in the 6TiSCH device was never concluded as the group focussed on best effort traffic. This work would be revived by RAW: Track Forwarding

By forwarding, this specification means the per-packet operation that allows to deliver a packet to a next hop or an upper layer in this node. Forwarding is based on pre-existing state that was installed as a result of the routing computation of a Track by a PCE. The 6TiSCH architecture supports three different forwarding model, G-MPLS Track Forwarding (TF), 6LoWPAN Fragment Forwarding (FF) and IPv6 Forwarding (6F) which is the classical IP operation [I-D.ietf-6tisch-architecture]. The DetNet case relates to the Track Forwarding operation under the control of a PCE.

A Track is a unidirectional path between a source and a destination. In a Track cell, the normal operation of IEEE802.15.4 Automatic Repeat-reQuest (ARQ) usually happens, though the acknowledgment may be omitted in some cases, for instance if there is no scheduled cell for a retry.

Track Forwarding is the simplest and fastest. A bundle of cells set to receive (RX-cells) is uniquely paired to a bundle of cells that are set to transmit (TX-cells), representing a layer-2 forwarding state that can be used regardless of the network layer protocol. This model can effectively be seen as a Generalized Multi-protocol Label Switching (G-MPLS) operation in that the information used to switch a frame is not an explicit label, but rather related to other properties of the way the packet was received, a particular cell in the case of 6TiSCH. As a result, as long as the TSCH MAC (and Layer-2 security) accepts a frame, that frame can be switched regardless of the protocol, whether this is an IPv6 packet, a 6LoWPAN fragment, or a frame from an alternate protocol such as WirelessHART or ISA100.11a.

A data frame that is forwarded along a Track normally has a destination MAC address that is set to broadcast - or a multicast address depending on MAC support. This way, the MAC layer in the intermediate nodes accepts the incoming frame and 6top switches it without incurring a change in the MAC header. In the case of IEEE802.15.4, this means effectively broadcast, so that along the Track the short address for the destination of the frame is set to 0xFFFF.

A Track is thus formed end-to-end as a succession of paired bundles, a receive bundle from the previous hop and a transmit bundle to the next hop along the Track, and a cell in such a bundle belongs to at most one Track. For a given iteration of the device schedule, the effective channel of the cell is obtained by adding a pseudo-random number to the channelOffset of the cell, which results in a rotation of the frequency that used for transmission. The bundles may be computed so as to accommodate both variable rates and retransmissions, so they might not be fully used at a given iteration of the schedule. The 6TiSCH architecture provides additional means to avoid waste of cells as well as overflows in the transmit bundle, as follows:

In one hand, a TX-cell that is not needed for the current iteration may be reused opportunistically on a per-hop basis for routed packets. When all of the frame that were received for a given Track are effectively transmitted, any available TX-cell for that Track can be reused for upper layer traffic for which the next-hop router matches the next hop along the Track. In that case, the cell that is being used is effectively a TX-cell from the Track, but the short address for the destination is that of the next-hop router. It results that a frame that is received in a RX-cell of a Track with a destination MAC address set to this node as opposed to broadcast must be extracted from the Track and delivered to the upper layer (a frame with an unrecognized MAC address is dropped at the lower MAC layer and thus is not received at the 6top sublayer).

On the other hand, it might happen that there are not enough TX-cells in the transmit bundle to accommodate the Track traffic, for instance if more retransmissions are needed than provisioned. In that case, the frame can be placed for transmission in the bundle that is used for layer-3 traffic towards the next hop along the Track as long as it can be routed by the upper layer, that is, typically, if the frame transports an IPv6 packet. The MAC address should be set to the next-hop MAC address to avoid confusion. It results that a frame that is received over a layer-3 bundle may be in fact associated to a Track. In a classical IP link such as an Ethernet, off-Track traffic is typically in excess over reservation to be routed along the non-reserved path based on its QoS setting. But with 6TiSCH, since the use of the layer-3 bundle may be due to transmission failures, it makes sense for the receiver to recognize a frame that should be re-Tracked, and to place it back on the appropriate bundle if possible. A frame should be re-Tracked if the Per-Hop-Behavior group indicated in the Differentiated Services Field in the IPv6 header is set to Deterministic Forwarding, as discussed in Section A frame is re-Tracked by scheduling it for transmission over the transmit bundle associated to the Track, with the destination MAC address set to broadcast.

There are 2 modes for a Track, transport mode and tunnel mode. Transport Mode

In transport mode, the Protocol Data Unit (PDU) is associated with flow-dependant meta-data that refers uniquely to the Track, so the 6top sublayer can place the frame in the appropriate cell without ambiguity. In the case of IPv6 traffic, this flow identification is transported in the Flow Label of the IPv6 header. Associated with the source IPv6 address, the Flow Label forms a globally unique identifier for that particular Track that is validated at egress before restoring the destination MAC address (DMAC) and punting to the upper layer.

                       |                                    ^
   +--------------+    |                                    |
   |     IPv6     |    |                                    |
   +--------------+    |                                    |
   |  6LoWPAN HC  |    |                                    |
   +--------------+  ingress                              egress
   |     6top     |   sets     +----+          +----+     restores
   +--------------+  dmac to   |    |          |    |     dmac to
   |   TSCH MAC   |   brdcst   |    |          |    |      self
   +--------------+    |       |    |          |    |       |
   |   LLN PHY    |    +-------+    +--...-----+    +-------+

Figure 7: Track Forwarding, Transport Mode Tunnel Mode

In tunnel mode, the frames originate from an arbitrary protocol over a compatible MAC that may or may not be synchronized with the 6TiSCH network. An example of this would be a router with a dual radio that is capable of receiving and sending WirelessHART or ISA100.11a frames with the second radio, by presenting itself as an Access Point or a Backbone Router, respectively.

In that mode, some entity (e.g. PCE) can coordinate with a WirelessHART Network Manager or an ISA100.11a System Manager to specify the flows that are to be transported transparently over the Track.

   |     IPv6     |
   |  6LoWPAN HC  |
   +--------------+             set            restore
   |     6top     |            +dmac+          +dmac+
   +--------------+          to|brdcst       to|nexthop
   |   TSCH MAC   |            |    |          |    |
   +--------------+            |    |          |    |
   |   LLN PHY    |    +-------+    +--...-----+    +-------+
   +--------------+    |   ingress                 egress   |
                       |                                    |
   +--------------+    |                                    |
   |   LLN PHY    |    |                                    |
   +--------------+    |                                    |
   |   TSCH MAC   |    |                                    |
   +--------------+    | dmac =                             | dmac =
   |ISA100/WiHART |    | nexthop                            v nexthop

Figure 8: Track Forwarding, Tunnel Mode

In that case, the flow information that identifies the Track at the ingress 6TiSCH router is derived from the RX-cell. The dmac is set to this node but the flow information indicates that the frame must be tunneled over a particular Track so the frame is not passed to the upper layer. Instead, the dmac is forced to broadcast and the frame is passed to the 6top sublayer for switching.

At the egress 6TiSCH router, the reverse operation occurs. Based on metadata associated to the Track, the frame is passed to the appropriate link layer with the destination MAC restored. Tunnel Metadata

Metadata coming with the Track configuration is expected to provide the destination MAC address of the egress endpoint as well as the tunnel mode and specific data depending on the mode, for instance a service access point for frame delivery at egress. If the tunnel egress point does not have a MAC address that matches the configuration, the Track installation fails.

In transport mode, if the final layer-3 destination is the tunnel termination, then it is possible that the IPv6 address of the destination is compressed at the 6LoWPAN sublayer based on the MAC address. It is thus mandatory at the ingress point to validate that the MAC address that was used at the 6LoWPAN sublayer for compression matches that of the tunnel egress point. For that reason, the node that injects a packet on a Track checks that the destination is effectively that of the tunnel egress point before it overwrites it to broadcast. The 6top sublayer at the tunnel egress point reverts that operation to the MAC address obtained from the tunnel metadata. OAM

An Overview of Operations, Administration, and Maintenance (OAM) Tools provides an overwiew of the existing tooling for OAM [RFC6291]. Tracks are complex paths and new tooling is necessary to manage them, with respect to load control, timing, and the Packet Replication and Elimination Functions (PREF).

An example of such tooling can be found in the context of BIER and more specifically BIER Traffic Engineering (BIER-TE): [I-D.thubert-bier-replication-elimination] leverages BIER-TE to control the process of PREF, and to provide traceability of these operations, in the deterministic dataplane, along a complex Track. For the 6TiSCH type of constrained environment, [I-D.thubert-6lo-bier-dispatch] enables an efficient encoding of the BIER bitmap within the 6LoRH framework.

6. 3GPP Ultra-Reliable Low-Latency Communication

coming up

7. L-band Digital Aeronautical Communications System

One of the main pillars of the modern Air Traffic Management (ATM) system is the existence of a communication infrastructure that enables efficient aircraft guidance and safe separation in all phases of flight. Although current systems are technically mature, they are suffering from the VHF band’s increasing saturation in high-density areas and the limitations posed by analogue radio. Therefore, aviation globally and the European Union (EU) in particular, strives for a sustainable modernization of the aeronautical communication infrastructure.

In the long-term, ATM communication shall transition from analogue VHF voice and VDL2 communication to more spectrum efficient digital data communication. The European ATM Master Plan foresees this transition to be realized for terrestrial communications by the development and implementation of the L-band Digital Aeronautical Communications System (LDACS). LDACS shall enable IPv6 based air-ground communication related to the safety and regularity of the flight. The particular challenge is that no new frequencies can be made available for terrestrial aeronautical communication. It was thus necessary to develop procedures to enable the operation of LDACS in parallel with other services in the same frequency band.

7.1. Provenance and Documents

The development of LDACS has already made substantial progress in the Single European Sky ATM Research (SESAR) framework, and is currently being continued in the follow-up program, SESAR2020 [RIH18]. A key objective of the SESAR activities is to develop, implement and validate a modern aeronautical data link able to evolve with aviation needs over long-term. To this end, an LDACS specification has been produced [GRA19] and is continuously updated; transmitter demonstrators were developed to test the spectrum compatibility of LDACS with legacy systems operating in the L-band [SAJ14]; and the overall system performance was analyzed by computer simulations, indicating that LDACS can fulfil the identified requirements [GRA11].

LDACS standardization within the framework of the International Civil Aviation Organization (ICAO) started in December 2016. The ICAO standardization group has produced an initial Standards and Recommended Practices (SARPs) document [ICAO18]. The SARPs document defines the general characteristics of LDACS. The ICAO standardization group plans to produce an ICAO technical manual - the ICAO equivalent to a technical standard - within the next years. Generally, the group is open to input from all sources and develops LDACS in the open.

Up to now the LDACS standardization has been focused on the development of the physical layer and the data link layer, only recently have higher layers come into the focus of the LDACS development activities. There is currently no "IPv6 over LDACS" specification; however, SESAR2020 has started the testing of IPv6-based LDACS testbeds. The IPv6 architecture for the aeronautical telecommunication network is called the Future Communications Infrastructure (FCI). FCI shall support quality of service, diversity, and mobility under the umbrella of the "multi-link concept". This work is conducted by ICAO working group WG-I.

In addition to standardization activities several industrial LDACS prototypes have been built. One set of LDACS prototypes has been evaluated in flight trials confirming the theoretical results predicting the system performance [GRA18][SCH19].

7.2. General Characteristics

LDACS will become one of several wireless access networks connecting aircraft to the Aeronautical Telecommunications Network (ATN). The LDACS access network contains several ground stations, each of them providing one LDACS radio cell. The LDACS air interface is a cellular data link with a star-topology connecting aircraft to ground-stations with a full duplex radio link. Each ground-station is the centralized instance controlling all air-ground communications within its radio cell. A ground-station supports up to 512 aircraft.

The LDACS air interface protocol stack defines two layers, the physical layer and the data link layer.

The physical layer provides the means to transfer data over the radio channel. The LDACS ground-station supports bi-directional links to multiple aircraft under its control. The forward link direction (FL; ground-to-air) and the reverse link direction (RL; air-to-ground) are separated by frequency division duplex. Forward link and reverse link use a 500 kHz channel each. The ground-station transmits a continuous stream of OFDM symbols on the forward link. In the reverse link different aircraft are separated in time and frequency using a combination of Orthogonal Frequency-Division Multiple-Access (OFDMA) and Time-Division Multiple-Access (TDMA). Aircraft thus transmit discontinuously on the reverse link with radio bursts sent in precisely defined transmission opportunities allocated by the ground-station. LDACS does not support beam-forming or Multiple Input Multiple Output (MIMO).

The data-link layer provides the necessary protocols to facilitate concurrent and reliable data transfer for multiple users. The LDACS data link layer is organized in two sub-layers: The medium access sub-layer and the logical link control sub-layer. The medium access sub-layer manages the organization of transmission opportunities in slots of time and frequency. The logical link control sub-layer provides acknowledged point-to-point logical channels between the aircraft and the ground-station using an automatic repeat request protocol. LDACS supports also unacknowledged point-to-point channels and ground-to-air broadcast.

The user data rate of LDACS is 315 kbit/s to 1428 kbit/s on the forward link, and 294 kbit/s to 1390 kbit/s on the reverse link, depending on coding and modulation. Due to strong interference from legacy systems in the L-band, the most robust coding and modulation should be expected for initial deployment i.e. 315/294 kbit/s on the forward/reverse link, respectively.

Since LDACS has been mainly designed for air traffic management communication it supports mutual entity authentication, integrity and confidentiality capabilities of user data messages and some control channel protection capabilities [MAE19].

7.3. Applicability to Deterministic Flows

LDACS has been designed with applications related to the safety and regularity of the flight in mind. It has therefore been designed as a deterministic wireless data link (as far as possible).

LDACS medium access is always under the control of the ground-station of a radio cell. Any medium access for the transmission of user data has to be requested with a resource request message stating the requested amount of resources and class of service. The ground-station performs resource scheduling on the basis of these requests and grants resources with resource allocation messages. Resource request and allocation messages are exchanged over dedicated contention-free control channels.

LDACS has two mechanisms to request resources from the scheduler in the ground-station. Resources can either be requested "on demand" with a given class of service. On the forward link, this is done locally in the ground-station, on the reverse link a dedicated contention-free control channel is used (Dedicated Control Channel (DCCH); roughly 83 bit every 60 ms). A resource allocation is always announced in the control channel of the forward link (Common Control Channel (CCCH); variable sized). Due to the spacing of the reverse link control channels of every 60 ms, a medium access delay in the same order of magnitude is to be expected.

Resources can also be requested "permanently". The permanent resource request mechanism supports requesting recurring resources in given time intervals. A permanent resource request has to be canceled by the user (or by the ground-station, which is always in control). User data transmissions over LDACS are therefore always scheduled by the ground-station, while control data uses statically (i.e. at net entry) allocated recurring resources (DCCH and CCCH). The current specification documents specify no scheduling algorithm. However performance evaluations so far have used strict priority scheduling and round robin for equal priorities for simplicity. In the current prototype implementations LDACS classes of service are thus realized as priorities of medium access and not as flows. Note that this can starve out low priority flows. However, this is not seen as a big problem since safety related message always go first in any case. Scheduling of reverse link resources is done in physical Protocol Data Units (PDU) of 112 bit (or larger if more aggressive coding and modulation is used). Scheduling on the forward link is done Byte-wise since the forward link is transmitted continuously by the ground-station.

In order to support diversity, LDACS supports handovers to other ground-stations on different channels. Handovers may be initiated by the aircraft (break-before-make) or by the ground-station (make-before-break) if it is connected to an alternative ground-station via the same ground-station controller. Beyond this, FCI diversity shall be implemented by the multi-link concept.

8. IANA Considerations

This specification does not require IANA action.

9. Security Considerations

Most RAW technologies integrate some authentication or encryption mechanisms that were defined outside the IETF.

10. Contributors

Georgios Z. Papadopoulos:
Contributed to the TSCH section.
Nils Mäurer:
Contributed to the LDACS section.
Thomas Gräupl:
Contributed to the LDACS section.

11. Acknowledgments

Many thanks to the participants of the RAW WG where a lot of the work discussed here happened.

12. References

12.1. Normative References

[I-D.ietf-6tisch-architecture] Thubert, P., "An Architecture for IPv6 over the TSCH mode of IEEE 802.15.4", Internet-Draft draft-ietf-6tisch-architecture-23, June 2019.
[I-D.ietf-detnet-architecture] Finn, N., Thubert, P., Varga, B. and J. Farkas, "Deterministic Networking Architecture", Internet-Draft draft-ietf-detnet-architecture-13, May 2019.
[RFC5673] Pister, K., Thubert, P., Dwars, S. and T. Phinney, "Industrial Routing Requirements in Low-Power and Lossy Networks", RFC 5673, DOI 10.17487/RFC5673, October 2009.
[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017.
[RFC8480] Wang, Q., Vilajosana, X. and T. Watteyne, "6TiSCH Operation Sublayer (6top) Protocol (6P)", RFC 8480, DOI 10.17487/RFC8480, November 2018.

12.2. Informative References

[Cavalcanti_2019] Dave Cavalcanti et al., "Extending Time Distribution and Timeliness Capabilities over the Air to Enable Future Wireless Industrial Automation Systems, the Proceedings of IEEE", June 2019.
[CCAMP] IETF, "Common Control and Measurement Plane"
[dearmas16] Jesica de Armas et al., "Determinism through path diversity: Why packet replication makes sense", September 2016.
[Ghasempour_2017] Yasaman Ghasempour et al., "802.11ay: Next-Generation 60 GHz Communications for 100 Gb/s Wi-Fi", December 2017.
[GRA11] Gräupl, T. and M. Ehammer, "L-DACS1 Data Link Layer Evolution of ATN/IPS", Proceedings of the 30th IEEE/AIAA Digital Avionics Systems Conference (DASC) Seattle, WA, USA, October 2011.
[GRA18] al., T. G. E., "L-band Digital Aeronautical Communications System (LDACS) flight trials in the national German project MICONAV", Proceedings of the Integrated Communications, Navigation, Surveillance Conference (ICNS) Herndon, VA, USA, April 2018.
[GRA19] Gräupl, T., Rihacek, C. and B. Haindl, "LDACS A/G Specification", SESAR2020 PJ14-02-01 D3.3.010, February 2019.
[I-D.ietf-6tisch-coap] Sudhaakar, R. and P. Zand, "6TiSCH Resource Management and Interaction using CoAP", Internet-Draft draft-ietf-6tisch-coap-03, March 2015.
[I-D.ietf-6tisch-msf] Chang, T., Vucinic, M., Vilajosana, X., Duquennoy, S. and D. Dujovne, "6TiSCH Minimal Scheduling Function (MSF)", Internet-Draft draft-ietf-6tisch-msf-03, April 2019.
[I-D.ietf-bier-te-arch] Eckert, T., Cauchie, G., Braun, W. and M. Menth, "Traffic Engineering for Bit Index Explicit Replication (BIER-TE)", Internet-Draft draft-ietf-bier-te-arch-02, May 2019.
[I-D.ietf-roll-nsa-extension] Koutsiamanis, R., Papadopoulos, G., Montavont, N. and P. Thubert, "RPL DAG Metric Container Node State and Attribute object type extension", Internet-Draft draft-ietf-roll-nsa-extension-03, June 2019.
[I-D.papadopoulos-paw-pre-reqs] Papadopoulos, G., Koutsiamanis, R., Montavont, N. and P. Thubert, "Exploiting Packet Replication and Elimination in Complex Tracks in LLNs", Internet-Draft draft-papadopoulos-paw-pre-reqs-01, March 2019.
[I-D.svshah-tsvwg-deterministic-forwarding] Shah, S. and P. Thubert, "Deterministic Forwarding PHB", Internet-Draft draft-svshah-tsvwg-deterministic-forwarding-04, August 2015.
[I-D.thubert-6lo-bier-dispatch] Thubert, P., Brodard, Z., Jiang, H. and G. Texier, "A 6loRH for BitStrings", Internet-Draft draft-thubert-6lo-bier-dispatch-06, January 2019.
[I-D.thubert-bier-replication-elimination] Thubert, P., Eckert, T., Brodard, Z. and H. Jiang, "BIER-TE extensions for Packet Replication and Elimination Function (PREF) and OAM", Internet-Draft draft-thubert-bier-replication-elimination-03, March 2018.
[ICAO18] International Civil Aviation Organization (ICAO), "L-Band Digital Aeronautical Communication System (LDACS)", International Standards and Recommended Practices Annex 10 - Aeronautical Telecommunications, Vol. III – Communication Systems, July 2018.
[IEEE80211] "IEEE Standard 802.11 - IEEE Standard for Information Technology - Telecommunications and information exchange between systems Local and metropolitan area networks - Specific requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications."
[IEEE80211ad] "802.11ad: Enhancements for very high throughput in the 60 GHz band"
[IEEE80211ak] "802.11ak: Enhancements for Transit Links Within Bridged Networks", 2017.
[IEEE80211ax] "802.11ax D4.0: Enhancements for High Efficiency WLAN"
[IEEE80211ay] "802.11ay: Enhanced throughput for operation in license-exempt bands above 45 GHz"
[IEEE80211be] "802.11be: Extreme High Throughput"
[IEEE802154] IEEE standard for Information Technology, "IEEE Std. 802.15.4, Part. 15.4: Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for Low-Rate Wireless Personal Area Networks"
[IEEE8021Qat] "802.1Qat: Stream Reservation Protocol"
[IEEE8021Qcc] "802.1Qcc: IEEE Standard for Local and Metropolitan Area Networks--Bridges and Bridged Networks -- Amendment 31: Stream Reservation Protocol (SRP) Enhancements and Performance Improvements"
[IEEE_doc_11-18-2009-06] "802.11 Real-Time Applications (RTA) Topic Interest Group (TIG) Report", November 2018.
[IEEE_doc_11-19-0373-00] Kevin Stanton et Al., "Time-Sensitive Applications Support in EHT", March 2019.
[ISA100.11a] ISA/IEC, "ISA100.11a, Wireless Systems for Automation, also IEC 62734", 2011.
[MAE19] Mäurer, N. and C. Schmitt, "DLR tests digital communications technologies combined with additional navigation functions for the first time", Proceedings of the Integrated Communications, Navigation, Surveillance Conference (ICNS) Washington D.C., USA, April 2019.
[morell13] Antoni Morell et al., "Label switching over IEEE802.15.4e networks", April 2013.
[Nitsche_2015] Thomas Nitsche et al., "IEEE 802.11ad: directional 60 GHz communication for multi-Gigabit-per-second Wi-Fi", December 2014.
[PCE] IETF, "Path Computation Element"
[RFC6291] Andersson, L., van Helvoort, H., Bonica, R., Romascanu, D. and S. Mansfield, "Guidelines for the Use of the "OAM" Acronym in the IETF", BCP 161, RFC 6291, DOI 10.17487/RFC6291, June 2011.
[RFC6550] Winter, T., Thubert, P., Brandt, A., Hui, J., Kelsey, R., Levis, P., Pister, K., Struik, R., Vasseur, JP. and R. Alexander, "RPL: IPv6 Routing Protocol for Low-Power and Lossy Networks", RFC 6550, DOI 10.17487/RFC6550, March 2012.
[RFC6551] Vasseur, JP., Kim, M., Pister, K., Dejean, N. and D. Barthel, "Routing Metrics Used for Path Calculation in Low-Power and Lossy Networks", RFC 6551, DOI 10.17487/RFC6551, March 2012.
[RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E. and Y. Weingarten, "An Overview of Operations, Administration, and Maintenance (OAM) Tools", RFC 7276, DOI 10.17487/RFC7276, June 2014.
[RFC8279] Wijnands, IJ., Rosen, E., Dolganow, A., Przygienda, T. and S. Aldrin, "Multicast Using Bit Index Explicit Replication (BIER)", RFC 8279, DOI 10.17487/RFC8279, November 2017.
[RIH18] Rihacek, C., Haindl, B., Fantappie, P., Pierattelli, S., Gräupl, T., Schnell, M. and N. Fistas, "L-band Digital Aeronautical Communications System (LDACS) Activities in SESAR2020", Proceedings of the Integrated Communications Navigation and Surveillance Conference (ICNS) Herndon, VA, USA, April 2018.
[SAJ14] Sajatovic, M., Günzel, H. and S. Müller, "WA04 D22 Test Report for Assessing LDACS1 Transmitter Impact upon DME/TACAN Receivers", April 2014.
[SCH19] Schnell, M., "DLR tests digital communications technologies combined with additional navigation functions for the first time", March 2019.
[TiSCH] IETF, "IPv6 over the TSCH mode over 802.15.4"
[vilajosana19] Xavier Vilajosana et al., "6TiSCH: Industrial Performance for IPv6 Internet-of-Things Networks", June 2019.
[WirelessHART] www.hartcomm.org, "Industrial Communication Networks - Wireless Communication Network and Communication Profiles - WirelessHART - IEC 62591", 2010.

Authors' Addresses

Pascal Thubert (editor) Cisco Systems, Inc Building D 45 Allee des Ormes - BP1200 MOUGINS - Sophia Antipolis, 06254 FRANCE Phone: +33 497 23 26 34 EMail: pthubert@cisco.com
Dave Cavalcanti Intel Corporation 2111 NE 25th Ave Hillsboro, OR, 97124 USA Phone: 503 712 5566 EMail: dave.cavalcanti@intel.com
Xavier Vilajosana Universitat Oberta de Catalunya 156 Rambla Poblenou Barcelona, Catalonia 08018 Spain EMail: xvilajosana@uoc.edu
Corinna Schmitt Universität der Bundeswehr München Werner-Heisenberg-Weg 39 Neubiberg, 85577 Germany EMail: corinna.schmitt@unibw.de