< draft-bhattacharyya-core-a-realist-00.txt   draft-bhattacharyya-core-a-realist-01.txt >
CoRE A. Bhattacharyya CoRE A. Bhattacharyya
Internet Draft S. Agrawal Internet Draft S. Agrawal
Intended status: Standards Track H. Rath Intended status: Standards Track H. Rath
Expires: April 2019 A. Pal Expires: August 2019 A. Pal
B. Purushothaman B. Purushothaman
TATA CONSULTANCY SERVICES LTD. TATA CONSULTANCY SERVICES LTD.
October 7, 2018 February 4, 2019
Adaptive RESTful Real-time Live Streaming for Things (A-REaLiST) Adaptive RESTful Real-time Live Streaming for Things (A-REaLiST)
draft-bhattacharyya-core-a-realist-00 draft-bhattacharyya-core-a-realist-01
Abstract Abstract
This draft presents extensions to Constrained Application Protocol This draft presents extensions to Constrained Application Protocol (CoAP) to
(CoAP) to enable RESTful Real-time Live Streaming for improving the enable RESTful Real-time Live Streaming for improving the Quality of Experience
Quality of Experience (QoE) for delay-sensitive Internet of Things (QoE) for delay-sensitive Internet of Things (IoT) applications. The overall
(IoT) applications. The overall architecture is termed ''Adaptive architecture is termed ''Adaptive RESTful Real-time Live Streaming for Things (A-
RESTful Real-time Live Streaming for Things (A-REaLiST)''. It is REaLiST)''. It is particularly designed for applications which rely on real-time
particularly designed for applications which rely on real-time augmented vision through live First Person View (FPV) feed from constrained remote
augmented vision through live First Person View (FPV) feed from agents like Unmanned Aerial Vehicle (UAV), etc. These extensions provide the
constrained remote agents like Unmanned Aerial Vehicle (UAV), etc. necessary hooks to help solution designers ensure low-latency transfer of streams
These extensions provide the necessary hooks to help solution and, for contents like video, a quick recovery from freeze and corruption without
designers ensure low-latency transfer of streams and, for contents incurring undue lag. A-REaLiST is an attempt to provide an integrated approach to
like video, a quick recovery from freeze and corruption without maintain the balance amongst QoE, resource-efficiency and loss resilience. It
incurring undue lag. A-REaLiST is an attempt to provide an provides the necessary hooks to optimize system performance by leveraging
integrated approach to maintain the balance amongst QoE, resource- contextual intelligence inferred from instantaneous information segments in
efficiency and loss resilience. It provides the necessary hooks to flight. These extensions equip CoAP with a standard for efficient RESTful
optimize system performance by leveraging contextual intelligence streaming for Internet of Things (IoT) contrary to HTTP-streaming in conventional
inferred from instantaneous information segments in flight. These Internet.
extensions equip CoAP with a standard for efficient RESTful
streaming for Internet of Things (IoT) contrary to HTTP-streaming in
conventional Internet.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the provisions of BCP 78
provisions of BCP 78 and BCP 79. and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are working documents of the Internet Engineering Task Force
months and may be updated, replaced, or obsoleted by other documents (IETF), its areas, and its working groups. Note that other groups may also
at any time. It is inappropriate to use Internet-Drafts as distribute working documents as Internet-Drafts.
reference material or to cite them other than as "work in progress."
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are draft documents valid for a maximum of six months and may be
Task Force (IETF), its areas, and its working groups. Note that updated, replaced, or obsoleted by other documents at any time. It is
other groups may also distribute working documents as Internet- inappropriate to use Internet-Drafts as reference material or to cite them other
Drafts. than as "work in progress."
Internet-Drafts are working documents of the Internet Engineering Task Force
(IETF), its areas, and its working groups. Note that other groups may also
distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six Internet-Drafts are draft documents valid for a maximum of six months and may be
months and may be updated, replaced, or obsoleted by other documents updated, replaced, or obsoleted by other documents at any time. It is
at any time. It is inappropriate to use Internet-Drafts as inappropriate to use Internet-Drafts as reference material or to cite them other
reference material or to cite them other than as "work in progress." than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on April 7, 2019. This Internet-Draft will expire on August 4, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the document authors.
document authors. All rights reserved. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating
Provisions Relating to IETF Documents to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of
(http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they
publication of this document. Please review these documents describe your rights and restrictions with respect to this document. Code
carefully, as they describe your rights and restrictions with Components extracted from this document must include Simplified BSD License text
respect to this document. Code Components extracted from this as described in Section 4.e of the Trust Legal Provisions and are provided without
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................3 1. Introduction ..................................................... 3
2. Revisiting CoAP................................................5 2. Revisiting CoAP................................................... 4
2.1. Some Interesting Aspects of CoAP..........................5 2.1. Some Interesting Aspects of CoAP ................................ 4
2.2. The Prevalent Approaches for Streaming over Internet......5 2.2. The Prevalent Approaches for Streaming over Internet ............... 5
2.3. CoAP as the Best of Two Worlds............................6 2.3. CoAP as the Best of Two Worlds ................................. 5
3. The Approach behind A-REaLiST..................................6 3. The Approach behind A-REaLiST ....................................... 5
3.1. Optional Context Aware Semantic Switch....................6 3.1. Optional Context Aware Semantic Switch ........................... 5
4. The Options Introduced.........................................7 4. The Options Introduced ............................................. 6
5. The Handshake and Exchange Semantics...........................8 5. The Handshake and Exchange Semantics ................................. 8
5.1. Initial Negotiation.......................................9 5.1. Initial Negotiation........................................... 8
5.2. Renegotiation............................................11 5.2. Renegotiation............................................... 10
6. Some Design Guidelines........................................13 6. Some Design Guidelines ............................................ 12
6.1. Implicit Congestion Avoidance............................13 6.1. Implicit Congestion Avoidance ................................. 12
6.2. Considerations for Consumer-side Rendering...............13 6.2. Considerations for Consumer-side Rendering ...................... 12
7. IANA Considerations...........................................14
8. Security Considerations.......................................14 7. IANA Considerations .............................................. 13
9. References....................................................14 8. Security Considerations ........................................... 13
9.1. Normative References.....................................14 9. References ...................................................... 13
9.2. Informative References...................................15 9.1. Normative References ......................................... 13
9.2. Informative References ....................................... 14
1. Introduction 1. Introduction
IoT emerged to facilitate exchange of frequent-but-small sensory IoT emerged to facilitate exchange of frequent-but-small sensory information
information amongst numerous constrained sensors [IOT- amongst numerous constrained sensors [IOT-ISOC][RFC7452]. However, recent trends
ISOC][RFC7452]. However, recent trends in industry and research in industry and research community realize the importance of live visual data as
community realize the importance of live visual data as important important sensory information. There are many discourses available to support this
sensory information. There are many discourses available to support observation [Murphy]. Live First Person View (FPV) from Unmanned Aerial Vehicles
this observation [Murphy]. Live First Person View (FPV) from (UAV) and dumb robot terminals are being used for futuristic remote control and
Unmanned Aerial Vehicles (UAV) and dumb robot terminals are being actuation applications for Augmented Reality (AR), Visual Simultaneous
used for futuristic remote control and actuation applications for Localization and Mapping (VSLAM), UAV based surveillance, etc. Efficacy of these
Augmented Reality (AR), Visual Simultaneous Localization and Mapping applications depends on resource-efficient, low-latency, yet high QoE transfer of
(VSLAM), UAV based surveillance, etc. Efficacy of these applications the FPV over the Internet (or IP networks in general). Contrary to the traditional
depends on resource-efficient, low-latency, yet high QoE transfer of video streaming applications, the UAV-like end-points (henceforth referred as
the FPV over the Internet (or IP networks in general). Contrary to 'video producer') that capture and transmit the FPV are resource constrained
the traditional video streaming applications, the UAV-like end- devices. Moreover, the producer may work in a lossy environment marred with
points (henceforth referred as 'video producer') that capture and fluctuating radio connectivity and disruptions due network congestion.
transmit the FPV are resource constrained devices. Moreover, the
producer may work in a lossy environment marred with fluctuating
radio connectivity and disruptions due network congestion.
The QoE considerations of the video rendering unit (henceforth The QoE considerations of the video rendering unit (henceforth referred as 'video
referred as 'video consumer') for these applications are quite consumer') for these applications are quite different from traditional
different from traditional applications. For example, in case of applications. For example, in case of highly delay sensitive AR applications, a
highly delay sensitive AR applications, a human brain may not human brain may not tolerate a noticeable video freeze or delayed reception, which
tolerate a noticeable video freeze or delayed reception, which might might have been overlooked for usual content delivery service like a YouTube
have been overlooked for usual content delivery service like a video. Such delay may result in wrong actuation. For example, delayed FPV from a
YouTube video. Such delay may result in wrong actuation. For UAV may lead to wrong control commands leading to catastrophic consequences. In
example, delayed FPV from a UAV may lead to wrong control commands addition, the communication should be as light-weight as possible to optimize the
leading to catastrophic consequences. In addition, the communication usage of on-board computing and energy resources of the UAV. So, real-time video
should be as light-weight as possible to optimize the usage of on- transmissions for IoT applications require special treatment [Pereira]. However,
board computing and energy resources of the UAV. So, real-time video as revealed through a detail analysis of the state-of-the-art in the next section,
transmissions for IoT applications require special treatment the existing solutions do not address such special requirements. This draft
[Pereira]. However, as revealed through a detail analysis of the attempts to bridge this important gap by extending CoAP [RFC7252].
state-of-the-art in the next section, the existing solutions do not
address such special requirements. This draft attempts to bridge
this important gap by extending CoAP [RFC7252].
To realize its purpose, the A-REaLiST architecture relies on To realize its purpose, the A-REaLiST architecture relies on [RFC7967] and adds
[RFC7967] and adds few new header options which, taken together, can few new header options which, taken together, can be conceived to form a
be conceived to form a conceptual 'Stream' extension on CoAP (Fig. conceptual 'Stream' extension on CoAP (Fig. 1).
1).
+----------------------+ +----------------------+
| Application | | Application |
+----------------------+ ---- +----------------------+ ----
| Stream | \ | Stream | \
|----------------------| \ |CoAP |----------------------| \ |CoAP
| Requests/Responses | | |extended | Requests/Responses | | |extended
|----------------------| | CoAP |for |----------------------| | CoAP |for
| Messages | | / A-REaLiST | Messages | | / A-REaLiST
+----------------------+ / ---- +----------------------+ / ----
+----------------------+ +----------------------+
| UDP | | UDP |
+----------------------+ +----------------------+
Figure 1: Abstract extended layering of CoAP for A-REaLiST with the Figure 1: Abstract extended layering of CoAP for A-REaLiST with the conceptual
conceptual layer for streaming. layer for streaming.
Though primarily designed for video streaming, these extensions can Though primarily designed for video streaming, these extensions can also be used
also be used to allow streaming of time-series information on CoAP. to allow streaming of time-series information on CoAP.
Note: Block-wise transfer [RFC7959] is a standardized extension to Note: Block-wise transfer [RFC7959] is a standardized extension to CoAP for
CoAP for transferring large application data. The cited use case transferring large application data. The cited use case for this is to perform
for this is to perform firmware upgrade for a large number of firmware upgrade for a large number of constrained devices. Block-wise
constrained devices. Block-wise transfer is primarily concerned transfer is primarily concerned with reliable delivery of information. It
with reliable delivery of information. It works in synchronized works in synchronized manner. If a message remains unacknowledged despite
manner. If a message remains unacknowledged despite retransmissions then the whole exchange is cancelled. So, it is not suitable
retransmissions then the whole exchange is cancelled. So, it is for real-time delivery [GIoTS] which is requirement for many time-series
not suitable for real-time delivery [GIoTS] which is requirement information streams including video.
for many time-series information streams including video.
2. Revisiting CoAP 2. Revisiting CoAP
2.1. Some Interesting Aspects of CoAP 2.1. Some Interesting Aspects of CoAP
( i) CoAP allows both confirmable (CON) and non-confirmable (NON) ( i) CoAP allows both confirmable (CON) and non-confirmable (NON) messaging.
messaging.
( ii) CON mode enables CoAP with an option for reliable RESTful ( ii) CON mode enables CoAP with an option for reliable RESTful delivery like HTTP
delivery like HTTP [RFC2616]on TCP. On the other hand, [RFC2616]on TCP. On the other hand, intelligent use of No-Response option
intelligent use of No-Response option [RFC7967] along with NON [RFC7967] along with NON mode can create an RTP like best-effort messaging on
mode can create an RTP like best-effort messaging on UDP. UDP.
(iii) Context based switching between the reliable and best-effort (iii) Context based switching between the reliable and best-effort semantics can
semantics can be executed from the end-application level. This be executed from the end-application level. This way an optimum balance
way an optimum balance between reliability delay-performance can between reliability delay-performance can be maintained to improve the overall
be maintained to improve the overall Quality of Experience (QoE). Quality of Experience (QoE).
( iv) The base CoAP specification is inherently designed for ( iv) The base CoAP specification is inherently designed for resource constrained
resource constrained devices. Hence, a streaming protocol using devices. Hence, a streaming protocol using the stateless RESTful semantics on
the stateless RESTful semantics on CoAP makes the solution CoAP makes the solution inherently lightweight. So, unlike conventional
inherently lightweight. So, unlike conventional approach the approach the designers can use a single stack that is equally efficient for
designers can use a single stack that is equally efficient for sending the small data out of sensors, as well as, infinite visual stream.
sending the small data out of sensors, as well as, infinite
visual stream.
2.2. The Prevalent Approaches for Streaming over Internet 2.2. The Prevalent Approaches for Streaming over Internet
The two prevalent approaches for streaming over the Internet are as The two prevalent approaches for streaming over the Internet are as below.
below.
First approach is to send the information segment over HTTP which First approach is to send the information segment over HTTP which uses the
uses the reliability feature of the underlying Transmission Control reliability feature of the underlying Transmission Control Protocol (TCP)
Protocol (TCP) transport. In this case TCP state-machine puts more transport. In this case TCP state-machine puts more emphasis on reliable delivery
emphasis on reliable delivery of segments rather than maintaining of segments rather than maintaining the real-time deadlines. However, this is
the real-time deadlines. However, this is right now the prevalent right now the prevalent approach as it treats video and other streams as general
approach as it treats video and other streams as general Internet Internet traffic. So, streaming can seamlessly co-exist with the existing Internet
traffic. So, streaming can seamlessly co-exist with the existing architecture. Also, since TCP takes care of ordered delivery, the end-application
Internet architecture. Also, since TCP takes care of ordered does not need to worry about these matters.
delivery, the end-application does not need to worry about these
matters.
The other approach is to use a specialized protocol like Real-time The other approach is to use a specialized protocol like Real-time Transport
Transport Protocol (RTP) [RFC3550]. It treats video and other real- Protocol (RTP) [RFC3550]. It treats video and other real-time streams as a special
time streams as a special type of traffic. To ensure real-time type of traffic. To ensure real-time delivery, the data is delivered in best-
delivery, the data is delivered in best-effort manner on top of UDP. effort manner on top of UDP. So, reliable delivery is undermined.
So, reliable delivery is undermined.
2.3. CoAP as the Best of Two Worlds 2.3. CoAP as the Best of Two Worlds
It can be conjectured, tallying the above with previous section, It can be conjectured, tallying the above with previous section, that CoAP
that CoAP inherently imbibes the functional features from HTTP-on- inherently imbibes the functional features from HTTP-on-TCP (reliable delivery)
TCP (reliable delivery) and RTP-on-UDP (best-effort delivery). and RTP-on-UDP (best-effort delivery). Further CoAP allows the switching between
Further CoAP allows the switching between these two seamlessly just these two seamlessly just by maneuvering the header options.
by maneuvering the header options.
3. The Approach behind A-REaLiST 3. The Approach behind A-REaLiST
The design stems from the principles of ''progressive download'' on The design stems from the principles of ''progressive download'' on top of the
top of the RESTful request/response semantics of CoAP. The RESTful request/response semantics of CoAP. The ''producer'' chunks the continuous
''producer'' chunks the continuous information stream into segments as information stream into segments as per the agreed maximum payload size suggested
per the agreed maximum payload size suggested in [RFC7252]. Each in [RFC7252]. Each chunk is transmitted as a CoAP request to a given resource at
chunk is transmitted as a CoAP request to a given resource at the the ''consumer''. This draft provides the necessary header extensions that enable
''consumer''. This draft provides the necessary header extensions that the ''consumer'' to maintain the sequence of the information segments in time and
enable the ''consumer'' to maintain the sequence of the information space.
segments in time and space.
3.1. Optional Context Aware Semantic Switch 3.1. Optional Context Aware Semantic Switch
Before forming the CoAP message for each segment, the streaming Before forming the CoAP message for each segment, the streaming application may
application may use a real-time analytics module (henceforth use a real-time analytics module (henceforth referred as 'analytics module') which
referred as 'analytics module') which may provide inference to the may provide inference to the ''Stream'' layer to decide the exchange semantics for
''Stream'' layer to decide the exchange semantics for the current the current segment. The message is sent reliably (CON message) or as best-effort
segment. The message is sent reliably (CON message) or as best- (NON message with No-Response option) based on the segment's information
effort (NON message with No-Response option) based on the segment's criticality. Criticality is measured in terms of importance of the segment-content
information criticality. Criticality is measured in terms of in reconstruction of the frames at the consumer. However, determination of
importance of the segment-content in reconstruction of the frames at criticality can be done on many aspects involving several application features
the consumer. However, determination of criticality can be done on like the source encoding type, the rendering logic at the consumer, etc. This way
many aspects involving several application features like the source the over-all balance between QoE and resource-consumption may be maintained. Fig.
encoding type, the rendering logic at the consumer, etc. This way 2 explains the idea with conceptual blocks. The overall concept and its efficacy
the over-all balance between QoE and resource-consumption may be has been explained with experimental results in [Wi-UAV-Globecom]
maintained. Fig. 2 explains the idea with conceptual blocks. The
overall concept and its efficacy has been explained with
experimental results in [Wi-UAV-Globecom]
+----------------------+ +----------------------+
| Application | Information segment --------- | Application | Information segment ---------
+----------------------+ ====================> |Real-time| +----------------------+ ====================> |Real-time|
| Stream | <==================== |Analytics| | Stream | <==================== |Analytics|
|----------------------| Reliable/ --------- |----------------------| Reliable/ ---------
| Requests/Responses | Best-effort? | Requests/Responses | Best-effort?
|----------------------| |----------------------|
| Messages | | Messages |
+----------------------+ +----------------------+
Figure 2: Illustrating the concept for context aware switching Figure 2: Illustrating the concept for context aware switching
Some examples are: Some examples are:
Example-1: Temporally compressed videos like MPEG consist of Group Example-1: Temporally compressed videos like MPEG consist of Group of Pictures
of Pictures (GoP) which comprises I-frames (Intra-frames) or key- (GoP) which comprises I-frames (Intra-frames) or key-frames, P-frames
frames, P-frames (Predicted frames) and B-frames (Bidirectional (Predicted frames) and B-frames (Bidirectional frames). Out of these 3 types
frames). Out of these 3 types of frames I-frames are most of frames I-frames are most critical in terms of synchronizing with the GoP at
critical in terms of synchronizing with the GoP at the receiver the receiver end for successful rendering. So, an analytics module at the
end for successful rendering. So, an analytics module at the ''video producer'' end may infer each information segments of I-frames as
''video producer'' end may infer each information segments of I- critical and send those segments reliably. The segments corresponding to P and
frames as critical and send those segments reliably. The segments B frames may be transferred as best-effort requests.
corresponding to P and B frames may be transferred as best-effort
requests.
Example-2: Let us consider a Motion JPEG (MJPEG) stream. In this Example-2: Let us consider a Motion JPEG (MJPEG) stream. In this case all the
case all the frames are independent JPEG frames and there is no frames are independent JPEG frames and there is no temporal compression. The
temporal compression. The analytics module may treat the segments analytics module may treat the segments containing MJPEG meta-data for each
containing MJPEG meta-data for each frame as critical segments frame as critical segments and transfer them through reliable messaging. Rest
and transfer them through reliable messaging. Rest of the of the segments may be transferred as best-effort requests. An intelligent
segments may be transferred as best-effort requests. An rendering engine at the ''consumer'' application may compensate for / conceal
intelligent rendering engine at the ''consumer'' application may any possible loss of non-meta-data (non-critical) segments using the reliably
compensate for / conceal any possible loss of non-meta-data (non- received meta-data and rest of the non-meta-data segments received through
critical) segments using the reliably received meta-data and rest best-effort. This way high QoE can be ensured despite reduced resource usage.
of the non-meta-data segments received through best-effort. This
way high QoE can be ensured despite reduced resource usage.
4. The Options Introduced 4. The Options Introduced
To achieve the purpose of the Stream layer, three new protocol To achieve the purpose of the Stream layer, three new protocol header options have
header options have been proposed as below: been proposed as below:
1) Stream_info: Consumes one unsigned byte. It maintains the stream
identity and indicates the present phase of exchange. It is both 1) Stream_info: Consumes one unsigned byte. It maintains the stream identity and
a request and response option. It has two fields. The 3-LSBs indicates the present phase of exchange. It is both a request and response
indicate the state of exchange (Stream_state) and 5-MSBs indicate option. It has two fields. The 3-LSBs indicate the state of exchange
an identifier (Stream_id) for the stream. The identifier remains (Stream_state) and 5-MSBs indicate an identifier (Stream_id) for the stream.
unchanged for the entire stream. So, The identifier remains unchanged for the entire stream. So,
Stream_id = Stream_info >> 3; Stream_id = Stream_info >> 3;
Stream_ state = Stream_info & 0x7. Stream_ state = Stream_info & 0x7.
Interpretation of Stream_state bits are : Interpretation of Stream_state bits are :
000=> stream initiation (always with request); 000=> stream initiation (always with request);
001=> initiation accepted (always with response); 001=> initiation accepted (always with response);
010=> initiation rejected (always with response); 010=> initiation rejected (always with response);
011=> stream re-negotiation (with request or response); 011=> stream re-negotiation (with request or response);
100=> stream ongoing. 100=> stream ongoing.
2) Time-stamp: It consumes 32-bit unsigned integer. It is a request 2) Time-stamp: It consumes 32-bit unsigned integer. It is a request option. It
option. It relates a particular application information segment relates a particular application information segment to the corresponding
to the corresponding frame in the play sequence. frame in the play sequence.
3) Position: It consumes 16-bit unsigned integer. It is a request 3) Position: It consumes 16-bit unsigned integer. It is a request option and MUST
option and MUST be accompanied with the Time-stamp option. It is be accompanied with the Time-stamp option. It is a combination of two fields.
a combination of two fields. The 15-MSBs indicate the ''offset'' at The 15-MSBs indicate the ''offset'' at which the present segment is placed in
which the present segment is placed in the frame corresponding to the frame corresponding to the given timestamp. The LSB indicates if the
the given timestamp. The LSB indicates if the current segment is current segment is the last segment of the frame corresponding to the given
the last segment of the frame corresponding to the given
timestamp. Hence, timestamp. Hence,
Last_segment = Position &0x01 ? True : False; Last_segment = Position &0x01 ? True : False;
Offset = (Position >> 1). Offset = (Position >> 1).
+-----+---+---+---+---+--------------+--------+--------+---------+ +-----+---+---+---+---+--------------+--------+--------+---------+
| No. | C | U | N | R | Name | Format | Length | Default | | No. | C | U | N | R | Name | Format | Length | Default |
+-----+---+---+---+---+--------------+--------+--------+---------+ +-----+---+---+---+---+--------------+--------+--------+---------+
| TBD | X | | - | | Stream-info | uint | 1 | (none) | | TBD | X | | - | | Stream-info | uint | 1 | (none) |
+-----+---+---+---+---+--------------+--------+--------+---------+ +-----+---+---+---+---+--------------+--------+--------+---------+
| TBD | X | | - | | Time-stamp | uint | 4 | (none) | | TBD | X | | - | | Time-stamp | uint | 4 | (none) |
+-----+---+---+---+---+--------------+--------+--------+---------+ +-----+---+---+---+---+--------------+--------+--------+---------+
| TBD | X | | - | | Position | uint | 2 | (none) | | TBD | X | | - | | Position | uint | 2 | (none) |
+-----+---+---+---+---+--------------+--------+--------+---------+ +-----+---+---+---+---+--------------+--------+--------+---------+
Table 1: Option Properties Table 1: Option Properties
5. The Handshake and Exchange Semantics 5. The Handshake and Exchange Semantics
As per the design considerations in view of the scenarios conceived As per the design considerations in view of the scenarios conceived at present,
at present, video transfer is initiated by the ''producer'' which acts video transfer is initiated by the ''producer'' which acts as the client.
as the client.
Note: The design considerations are driven by the experiences drawn Note: The design considerations are driven by the experiences drawn from the
from the applications where live video feeds are transmitted from applications where live video feeds are transmitted from battery operated
battery operated constrained ''video producers'' like UAVs and dumb constrained ''video producers'' like UAVs and dumb robotic terminals, etc. For
robotic terminals, etc. For example, while a fixed infrastructure example, while a fixed infrastructure system is using streamed FPV feed from UAVs,
system is using streamed FPV feed from UAVs, there may be situations there may be situations where each time a UAV is low on resources (energy and
where each time a UAV is low on resources (energy and computation, a computation, a new UAV with better state of resources (fresh battery, etc.) is
new UAV with better state of resources (fresh battery, etc.) is commissioned. The overall operation becomes simple if the newly commissioned UAV
commissioned. The overall operation becomes simple if the newly readily starts its job by streaming to the same resource at the fixed
commissioned UAV readily starts its job by streaming to the same infrastructure. It can be easily configured to determine whether the consumer is
resource at the fixed infrastructure. It can be easily configured to up and watching by observing the responses to the CON requests. In case the
determine whether the consumer is up and watching by observing the exchange is initiated by the consumer then whenever a new UAV is commissioned, the
responses to the CON requests. In case the exchange is initiated by consumer has to re-initiate the request again.
the consumer then whenever a new UAV is commissioned, the consumer
has to re-initiate the request again.
Each segment is transmitted to the ''video consumer'' as a POST Each segment is transmitted to the ''video consumer'' as a POST request. The Time-
request. The Time-stamp and Position options help sequential stamp and Position options help sequential ordering of the segments at the
ordering of the segments at the consumer. consumer.
5.1. Initial Negotiation 5.1. Initial Negotiation
Initial negotiations for frame rate, video type, encoding details, Initial negotiations for frame rate, video type, encoding details, etc., are
etc., are performed by exchanging configuration scripts (cbor or performed by exchanging configuration scripts (cbor or json) over POST request.
json) over POST request. Exact format of the script is application Exact format of the script is application dependent and is not part of this draft.
dependent and is not part of this draft.
Fig. 3 illustrates the exemplary exchanges related to handshakes for Fig. 3 illustrates the exemplary exchanges related to handshakes for connection
connection initiation. initiation.
Note: All reliable transfers are in blocking mode. So, the producer Note: All reliable transfers are in blocking mode. So, the producer MUST wait to
MUST wait to send any further segment (critical/ on-critical) till send any further segment (critical/ on-critical) till the response is received for
the response is received for the critical segment. Please refer to the critical segment. Please refer to Section 6 for suggested behavior in case a
Section 6 for suggested behavior in case a reliable transfer fails. reliable transfer fails.
Client (Producer) Server (Consumer) Client (Producer) Server (Consumer)
| | | |
| POST: CON; | | POST: CON; |
| URI=/video; | | URI=/video; |
| Stream-info = <5-bit ID>000; | | Stream-info = <5-bit ID>000; |
| Payload= CBOR or JSON |\ | Payload= CBOR or JSON |\
+------------------------------------------------->| | +------------------------------------------------->| |
| | |Stream | | |Stream
| ACK; | |negotiation | ACK; | |negotiation
skipping to change at page 11, line 4 skipping to change at page 10, line 4
| | | Stream | | | Stream
| POST: NON; | | ongoing | POST: NON; | | ongoing
| URI=/video; No-response = 127 | | | URI=/video; No-response = 127 | |
| Stream-info = <5-bit ID>100; | | | Stream-info = <5-bit ID>100; | |
| Time-stamp = <time_stamp_of_this_frame>; | | | Time-stamp = <time_stamp_of_this_frame>; | |
| Position = 1024; | | | Position = 1024; | |
| Payload= <Bytes_in 2nd _segment> | | | Payload= <Bytes_in 2nd _segment> | |
+------------------------------------------------->| | +------------------------------------------------->| |
| | | | | |
: : | : : |
Figure 3: Example showing successful negotiation of streaming Figure 3: Example showing successful negotiation of streaming parameters followed
parameters followed by transmission of video information and by transmission of video information and control. It is assumed that the segment
control. It is assumed that the segment size negotiated as 1024 at size negotiated as 1024 at the initiation. So, the position of the 2nd block is
the initiation. So, the position of the 2nd block is 1024. Note the 1024. Note the use of No-response option with NON request for the non-critical
use of No-response option with NON request for the non-critical
segment. segment.
5.2. Renegotiation 5.2. Renegotiation
The renegotiation phase may occur when the ''consumer'' does not agree The renegotiation phase may occur when the ''consumer'' does not agree to
to parameters proposed by the producer and proposes a modified set. parameters proposed by the producer and proposes a modified set. This may happen
This may happen when the consumer application may need a less frame- when the consumer application may need a less frame-rate than what is proposed by
rate than what is proposed by the producer. So, the ''consumer'' may the producer. So, the ''consumer'' may request a lower frame-rate and thereby avoid
request a lower frame-rate and thereby avoid unnecessary traffic in unnecessary traffic in the network. The reduction may also be driven by the
the network. The reduction may also be driven by the processing load processing load on the producer which is anyway a constrained device. So, if a
on the producer which is anyway a constrained device. So, if a consumer requests more frame-rate than what is initially proposed by the producer,
consumer requests more frame-rate than what is initially proposed by then the producer may insist on the lower frame-rate. Renegotiation may also occur
the producer, then the producer may insist on the lower frame-rate. if, during a stream, the producer senses a change in the end-to-end channel
Renegotiation may also occur if, during a stream, the producer condition and proposes a new set of best possible parameters that can be served
senses a change in the end-to-end channel condition and proposes a to the consumer.
new set of best possible parameters that can be served to the
consumer.
Note that, that the consumer is never allowed to exceed the limits Note that, that the consumer is never allowed to exceed the limits advertised by
advertised by the producer. the producer.
Fig. 4 illustrates exemplary exchanges for re-negotiation. Fig. 4 illustrates exemplary exchanges for re-negotiation.
Client (Producer) Server (Consumer) Client (Producer) Server (Consumer)
| | | |
| POST: CON; | | POST: CON; |
| URI=/video; | | URI=/video; |
| Stream-info = <5-bit ID>000; | | Stream-info = <5-bit ID>000; |
| Payload= CBOR or JSON |\ Initial | Payload= CBOR or JSON |\ Initial
+------------------------------------------------->| |negotiation +------------------------------------------------->| |negotiation
skipping to change at page 12, line 32 skipping to change at page 11, line 32
| Payload= CBOR or JSON |\ Successful | Payload= CBOR or JSON |\ Successful
+------------------------------------------------->| |renegotiation +------------------------------------------------->| |renegotiation
| | |as the | | |as the
| ACK; | |consumer | ACK; | |consumer
| Response = 2.04 CHANGED | |agrees to the | Response = 2.04 CHANGED | |agrees to the
| Steam-info = <5-bit ID>001 | |revised | Steam-info = <5-bit ID>001 | |revised
|<-------------------------------------------------|/ proposal. |<-------------------------------------------------|/ proposal.
: : : :
: (Streaming starts) : : (Streaming starts) :
Figure 4: Example showing successful renegotiation of streaming Figure 4: Example showing successful renegotiation of streaming parameters. Note
parameters. Note the maneuvering of the Stream-info bit patterns. the maneuvering of the Stream-info bit patterns.
Fig. 5 illustrates exemplary exchanges when a stream negotiation is Fig. 5 illustrates exemplary exchanges when a stream negotiation is unsuccessful.
unsuccessful. The accompanied script may provide hints to the reason The accompanied script may provide hints to the reason for unsuccessful
for unsuccessful negotiations. A simple case of unsuccessful attempt negotiations. A simple case of unsuccessful attempt may be observed if the
may be observed if the resource on the ''consumer'' side is not ready. resource on the ''consumer'' side is not ready. The exact formatting of the script
The exact formatting of the script is not in the scope of this is not in the scope of this draft.
draft.
Client (Producer) Server (Consumer) Client (Producer) Server (Consumer)
| | | |
| POST: CON; | | POST: CON; |
| URI=/video; | | URI=/video; |
| Stream-info = <5-bit ID>000; | | Stream-info = <5-bit ID>000; |
| Payload= CBOR or JSON |\ Unsuccessful | Payload= CBOR or JSON |\ Unsuccessful
+------------------------------------------------->| |negotiation. +------------------------------------------------->| |negotiation.
| | |The request | | |The request
| ACK; | |is successful. | ACK; | |is successful.
| Response = 2.04 CHANGED | |But consumer | Response = 2.04 CHANGED | |But consumer
| Steam-info = <5-bit ID>011 | |may reject | Steam-info = <5-bit ID>011 | |may reject
| Payload= CBOR or JSON | |for some | Payload= CBOR or JSON | |for some
|<-------------------------------------------------|/ reason |<-------------------------------------------------|/ reason
| | mentioned in | | mentioned in
Script. Script.
Figure 5: Example showing unsuccessful renegotiation despite Figure 5: Example showing unsuccessful renegotiation despite successful response
successful response code against the initiation request. code against the initiation request.
6. Some Design Guidelines 6. Some Design Guidelines
6.1. Implicit Congestion Avoidance 6.1. Implicit Congestion Avoidance
The throughput and resource optimization for A-REaLiST depends The throughput and resource optimization for A-REaLiST depends largely on the
largely on the best-effort delivery on UDP. Despite that the best-effort delivery on UDP. Despite that the application designer can make A-
application designer can make A-REaLiST implicitly congestion aware REaLiST implicitly congestion aware and proactively avoid congestion. CoAP has a
and proactively avoid congestion. CoAP has a basic congestion basic congestion avoidance mechanism which uses exponential back off to increase
avoidance mechanism which uses exponential back off to increase the the timeout for retransmissions. However, that works only for CON messages.
timeout for retransmissions. However, that works only for CON
messages.
The implicit congestion avoidance works like this: In case the The implicit congestion avoidance works like this: In case the producer fails to
producer fails to successfully transfer a critical segment of a successfully transfer a critical segment of a frame within the MAX_TRANSMIT_SPAN
frame within the MAX_TRANSMIT_SPAN as well as within MAX_RETRANSMIT as well as within MAX_RETRANSMIT [RFC7252] attempts, the producer drops
[RFC7252] attempts, the producer drops transmission of rest of the transmission of rest of the segments in that frame and waits for the next frame to
segments in that frame and waits for the next frame to be ready. The be ready. The rationale is, since the critical segment is not delivered, the
rationale is, since the critical segment is not delivered, the consumer will fail to reconstruct this frame anyway. So, there is no point in
consumer will fail to reconstruct this frame anyway. So, there is no clogging the network with rest of the segments.
point in clogging the network with rest of the segments.
6.2. Considerations for Consumer-side Rendering 6.2. Considerations for Consumer-side Rendering
While the critical segments are delivered reliably in a sequential While the critical segments are delivered reliably in a sequential manner, non-
manner, non-critical are delivered with best-effort in an open-loop critical are delivered with best-effort in an open-loop exchange. Also, the whole
exchange. Also, the whole frame can be dropped to avoid congestion. frame can be dropped to avoid congestion. Hence, the application at the
Hence, the application at the ''consumer'' end-point (server) needs to ''consumer'' end-point (server) needs to deal with issues like out-of-order
deal with issues like out-of-order delivery, frame/segment loss, delivery, frame/segment loss, asynchronous segment arrival.
asynchronous segment arrival.
The issues mentioned above have been discussed in literatures The issues mentioned above have been discussed in literatures [Perkins]. So the
[Perkins]. So the basic approach should be: Buffer till a critical basic approach should be: Buffer till a critical time to iron out the jittery,
time to iron out the jittery, out-of-order arrival of the segments, out-of-order arrival of the segments, play out from the appropriate buffer at a
play out from the appropriate buffer at a constant rate determined constant rate determined by the frame-rate of the video. There may be intelligent
by the frame-rate of the video. There may be intelligent algorithms algorithms to play-out with high QoE despite non-arrival of non-critical segments
to play-out with high QoE despite non-arrival of non-critical within the play-out deadline. This draft provides the hooks to create such
segments within the play-out deadline. This draft provides the hooks designs. Reference architecture of the play-out mechanism is provided in [Wi-UAV-
to create such designs. Reference architecture of the play-out Globecom]. The play-out architecture leverages on the design assumption about the
mechanism is provided in [Wi-UAV-Globecom]. The play-out 'less-constrained' nature of the consumer in terms of memory and processor.
architecture leverages on the design assumption about the 'less-
constrained' nature of the consumer in terms of memory and 6.3. Determining the segment size
processor.
Size of the information segment in a CoAP message should be limited by the least
possible MTU for the end-to-end channel. This is to ensure that there is no
undesired conversation state at the lower layers of the protocol stack due to
uncontrolled fragmentation leading to undesired explosion of traffic in the
network. For IPV6 network, the MTU can be determined using Path MTU Discovery
(PMTUD) [RFC8201] which bestows the responsibility of determining the path MTU on
the end-points itself.
The size of the segment should be guided by the recommendations as specified in
Section 4.6 of [RFC7252].
7. IANA Considerations 7. IANA Considerations
The IANA is requested to assign numbers to the three options The IANA is requested to assign numbers to the three options introduced in this
introduced in this draft for inclusion in the ''CoAP Option Numbers" draft for inclusion in the ''CoAP Option Numbers" registry as shown below.
registry as shown below.
+--------+--------------+-------------+ +--------+--------------+-------------+
| Number | Name | Reference | | Number | Name | Reference |
+--------+--------------+-------------+ +--------+--------------+-------------+
| TBD | Stream-info | Section 4 | | TBD | Stream-info | Section 4 |
+--------+--------------+-------------+ +--------+--------------+-------------+
| TBD | Time-stamp | Section 4 | | TBD | Time-stamp | Section 4 |
+--------+--------------+-------------+ +--------+--------------+-------------+
| TBD | Position | Section 4 | | TBD | Position | Section 4 |
+--------+--------------+-------------+ +--------+--------------+-------------+
8. Security Considerations 8. Security Considerations
This draft presents no security considerations beyond those in This draft presents no security considerations beyond those in Section 11 of the
Section 11 of the base CoAP specification [RFC7252]. base CoAP specification [RFC7252].
9. References 9. References
9.1. Normative References 9.1. Normative References
[RFC7252] [RFC7252]
Shelby, Z., Hartke, K. and Bormann, C.,"Constrained Application Protocol (CoAP)",
Shelby, Z., Hartke, K. and Bormann, C.,"Constrained Application RFC 7252, June, 2014.
Protocol (CoAP)", RFC 7252, June, 2014.
[RFC7967] [RFC7967]
Bhattacharyya, A., Bandyopadhyay, S., Pal, A., Bose, T.,
''Constrained Application Protocol (CoAP) Option for No Server Bhattacharyya, A., Bandyopadhyay, S., Pal, A., Bose, T., ''Constrained Application
Response'', RFC 7967, August, 2016. Protocol (CoAP) Option for No Server Response'', RFC 7967, August, 2016.
[RFC8201]
McCann, J., et al., ''Path MTU Discovery for IP version 6'', RFC 8201, July, 2017.
9.2. Informative References 9.2. Informative References
[IOT-ISOC] [IOT-ISOC]
Rose, K., Eldridge, S., Chapin, L., ''The Internet of Things: an Rose, K., Eldridge, S., Chapin, L., ''The Internet of Things: an overview'',
overview'', Internet Society, pp.1-50, October, 2015. Internet Society, pp.1-50, October, 2015.
[RFC7452] [RFC7452]
Tschofenig, H., Arkko, J., McPherson, D., "Architectural Tschofenig, H., Arkko, J., McPherson, D., "Architectural Considerations in Smart
Considerations in Smart Object Networking", RFC 7452, March, 2015. Object Networking", RFC 7452, March, 2015.
[Murphy] [Murphy]
Murphy, C., ''Internet of Things: Are you underestimating video?'', Murphy, C., ''Internet of Things: Are you underestimating video?'', Available
Available online: online:
http://www.informationweek.com/bigdata/bigdataanalytics/internetofth http://www.informationweek.com/bigdata/bigdataanalytics/internetofthingsareyouunde
ingsareyouunderestimatingvideo/a/d-id/1269508, June, 2014. restimatingvideo/a/d-id/1269508, June, 2014.
[Pereira] [Pereira]
Pereira, R., Pereira, E. G., ''Video Streaming Considerations for Pereira, R., Pereira, E. G., ''Video Streaming Considerations for Internet of
Internet of Things'', International Conference on Future Internet of Things'', International Conference on Future Internet of Things and Cloud, pp. 48-
Things and Cloud, pp. 48-52, August, 2014. 52, August, 2014.
[RFC7959] [RFC7959]
Bormann, C., Shelby, Z., ''Block-Wise Transfers in the Constrained Bormann, C., Shelby, Z., ''Block-Wise Transfers in the Constrained Application
Application Protocol (CoAP)'', RFC 7959, August, 2016. Protocol (CoAP)'', RFC 7959, August, 2016.
[GIoTS] [GIoTS]
Dey, S., Bhattacharyya, A., Mukherjee, A., "Semantic data exchange between
Dey, S., Bhattacharyya, A., Mukherjee, A., "Semantic data exchange collaborative robots in fog environment: Can CoAP be a choice?", Global IoTS, pp.
between collaborative robots in fog environment: Can CoAP be a 1-6, June, 2017.
choice?", Global IoTS, pp. 1-6, June, 2017.
[RFC2616] [RFC2616]
Fielding, R., Irvine, U.C., Gettys, J., Mogul, J., Frystyk, H., Fielding, R., Irvine, U.C., Gettys, J., Mogul, J., Frystyk, H., Masinter, L.,
Masinter, L., Leach, P., Berners-Lee, T., ''Hypertext Transfer Leach, P., Berners-Lee, T., ''Hypertext Transfer Protocol -- HTTP/1.1'', RFC 2616,
Protocol -- HTTP/1.1'', RFC 2616, June, 1999. June, 1999.
[RFC3550] [RFC3550]
Schulzrinne, H., Casner, S., Frederick, R., Jacobson, V., ''RTP: A Schulzrinne, H., Casner, S., Frederick, R., Jacobson, V., ''RTP: A Transport
Transport Protocol for Real-Time Applications'', RFC 3550, July, Protocol for Real-Time Applications'', RFC 3550, July, 2003.
2003.
[Wi-UAV-Globecom] [Wi-UAV-Globecom]
Bhattacharyya, A., Agrawal, S., Rath, H., Pal, A., ''Improving Live- Bhattacharyya, A., Agrawal, S., Rath, H., Pal, A., ''Improving Live-streaming
streaming Experience for Delay-sensitive IoT Applications : A Experience for Delay-sensitive IoT Applications : A RESTful Approach'', accepted in
RESTful Approach'', accepted in Globecom (Wi-UAV workshop), Dec., Globecom (Wi-UAV workshop), Dec., 2018.
2018.
[Perkins] [Perkins]
Perkins, C., ''RTP: Audio and Video for the Internet'', Addison- Perkins, C., ''RTP: Audio and Video for the Internet'', Addison-Wesley, 2003.
Wesley, 2003.
Authors' Addresses Authors' Addresses
Abhijan Bhattacharyya Abhijan Bhattacharyya
Tata Consultancy Services Ltd. Tata Consultancy Services Ltd.
Kolkata, India Kolkata, India
Email: abhijan.bhattacharyya@tcs.com Email: abhijan.bhattacharyya@tcs.com
Suvrat Agrawal Suvrat Agrawal
 End of changes. 63 change blocks. 
360 lines changed or deleted 325 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/