--- 1/draft-ietf-avtext-lrr-06.txt 2017-07-02 17:13:49.849431538 -0700 +++ 2/draft-ietf-avtext-lrr-07.txt 2017-07-02 17:13:49.881432311 -0700 @@ -1,22 +1,22 @@ Payload Working Group J. Lennox Internet-Draft D. Hong Intended status: Standards Track Vidyo -Expires: December 17, 2017 J. Uberti +Expires: December 31, 2017 J. Uberti S. Holmer M. Flodman Google - June 15, 2017 + June 29, 2017 The Layer Refresh Request (LRR) RTCP Feedback Message - draft-ietf-avtext-lrr-06 + draft-ietf-avtext-lrr-07 Abstract This memo describes the RTCP Payload-Specific Feedback Message "Layer Refresh Request" (LRR), which can be used to request a state refresh of one or more substreams of a layered media stream. It also defines its use with several RTP payloads for scalable media formats. Status of This Memo @@ -26,21 +26,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on December 17, 2017. + This Internet-Draft will expire on December 31, 2017. Copyright Notice Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -49,34 +49,34 @@ include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 2 2.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3 3. Layer Refresh Request . . . . . . . . . . . . . . . . . . . . 5 - 3.1. Message Format . . . . . . . . . . . . . . . . . . . . . 5 + 3.1. Message Format . . . . . . . . . . . . . . . . . . . . . 6 3.2. Semantics . . . . . . . . . . . . . . . . . . . . . . . . 7 - 4. Usage with specific codecs . . . . . . . . . . . . . . . . . 7 - 4.1. H264 SVC . . . . . . . . . . . . . . . . . . . . . . . . 7 + 4. Usage with specific codecs . . . . . . . . . . . . . . . . . 8 + 4.1. H264 SVC . . . . . . . . . . . . . . . . . . . . . . . . 8 4.2. VP8 . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 - 4.3. H265 . . . . . . . . . . . . . . . . . . . . . . . . . . 9 - 5. Usage with different scalability transmission mechanisms . . 10 - 6. Security Considerations . . . . . . . . . . . . . . . . . . . 11 - 7. SDP Definitions . . . . . . . . . . . . . . . . . . . . . . . 11 - 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 11 + 4.3. H265 . . . . . . . . . . . . . . . . . . . . . . . . . . 10 + 5. Usage with different scalability transmission mechanisms . . 11 + 6. SDP Definitions . . . . . . . . . . . . . . . . . . . . . . . 11 + 7. Security Considerations . . . . . . . . . . . . . . . . . . . 12 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 9.1. Normative References . . . . . . . . . . . . . . . . . . 12 9.2. Informative References . . . . . . . . . . . . . . . . . 13 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14 1. Introduction This memo describes an RTCP [RFC3550] Payload-Specific Feedback Message [RFC4585] "Layer Refresh Request" (LRR). It is designed to allow a receiver of a layered media stream to request that one or more of its substreams be refreshed, such that it can then be decoded by an endpoint which previously was not receiving those layers, without requiring that the entire stream be refreshed (as it would be if the receiver sent a Full Intra Request (FIR); [RFC5104] see also @@ -92,92 +92,96 @@ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 2.1. Terminology A "Layer Refresh Point" is a point in a scalable stream after which a decoder, which previously had been able to decode only some (possibly none) of the available layers of stream, is able to decode a greater number of the layers. - For spatial (or quality) layers, layer refresh typically requires - that a spatial layer be encoded in a way that references only lower- - layer subpictures of the current picture, not any earlier pictures of - that spatial layer. Additionally, the encoder must promise that no - earlier pictures of that spatial layer will be used as reference in - the future. + For spatial (or quality) layers, in normal encoding, a subpicture can + depend both on earlier pictures of that spatial layer and also on + lower-layer pictures of the current picture. A layer refresh, + however, typically requires that a spatial layer picture be encoded + in a way that references only the lower-layer subpictures of the + current picture, not any earlier pictures of that spatial layer. + Additionally, the encoder must promise that no earlier pictures of + that spatial layer will be used as reference in the future. - In a layer refresh, however, other layers than the ones requested for - refresh may still maintain dependency on earlier content of the + However, even in a layer refresh, layers other than the ones being + refreshed may still maintain dependency on earlier content of the stream. This is the difference between a layer refresh and a Full Intra Request [RFC5104]. This minimizes the coding overhead of refresh to only those parts of the stream that actually need to be refreshed at any given time. An illustration of spatial layer refresh of an enhancement layer is - shown below. + shown below. <-- indicates a coding dependency. ... <-- S1 <-- S1 S1 <-- S1 <-- ... | | | | \/ \/ \/ \/ ... <-- S0 <-- S0 <-- S0 <-- S0 <-- ... 1 2 3 4 Figure 1 In Figure 1, frame 3 is a layer refresh point for spatial layer S1; a decoder which had previously only been decoding spatial layer S0 would be able to decode layer S1 starting at frame 3. An illustration of spatial layer refresh of a base layer is shown - below. + below. <-- indicates a coding dependency. ... <-- S1 <-- S1 <-- S1 <-- S1 <-- ... | | | | \/ \/ \/ \/ ... <-- S0 <-- S0 S0 <-- S0 <-- ... 1 2 3 4 Figure 2 In Figure 2, frame 3 is a layer refresh point for spatial layer S0; a decoder which had previously not been decoding the stream at all could decode layer S0 starting at frame 3. - For temporal layers, layer refresh requires that the layer be - "temporally nested", i.e. use as reference only earlier frames of a - lower temporal layer, not any earlier frames of this temporal layer, - and also promise that no future frames of this temporal layer will - reference frames of this temporal layer before the refresh point. In - many cases, the temporal structure of the stream will mean that all - frames are temporally nested, in which case decoders will have no - need to send LRR messages for the stream. + For temporal layers, while normal encoding allows frames to depend on + earlier frames of the same temporal layer, layer refresh requires + that the layer be "temporally nested", i.e. use as reference only + earlier frames of a lower temporal layer, not any earlier frames of + this temporal layer, and also promise that no future frames of this + temporal layer will reference frames of this temporal layer before + the refresh point. In many cases, the temporal structure of the + stream will mean that all frames are temporally nested, in which case + decoders will have no need to send LRR messages for the stream. - An illustration of temporal layer refresh is shown below. + An illustration of temporal layer refresh is shown below. <-- + indicates a coding dependency. ... <----- T1 <------ T1 T1 <------ ... / / / |_ |_ |_ ... <-- T0 <------ T0 <------ T0 <------ T0 <--- ... 1 2 3 4 5 6 7 Figure 3 In Figure 3, frame 6 is a layer refresh point for temporal layer T1; a decoder which had previously only been decoding temporal layer T0 would be able to decode layer T1 starting at frame 6. An illustration of an inherently temporally nested stream is shown - below. + below. <-- indicates a coding dependency. T1 T1 T1 / / / |_ |_ |_ ... <-- T0 <------ T0 <------ T0 <------ T0 <--- ... 1 2 3 4 5 6 7 Figure 4 @@ -215,56 +219,57 @@ entry applies to a different media sender, identified by its SSRC. [NOTE TO RFC Editor: Please replace "TBD" with the IANA-assigned payload-specific feedback number.] 3.1. Message Format The Feedback Control Information (FCI) for the Layer Refresh Request consists of one or more FCI entries, the content of which is depicted in Figure 5. The length of the LRR feedback message MUST be set to - 2+3*N, where N is the number of FCI entries. + 2+3*N 32-bit words, where N is the number of FCI entries. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seq nr. |C| Payload Type| Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - | RES | TTID| TLID | RES | CTID| CLID (opt) | + | RES | TTID| TLID | RES | CTID| CLID | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 5 SSRC (32 bits) The SSRC value of the media sender that is requested to send a layer refresh point. Seq nr. (8 bits) Command sequence number. The sequence number space is unique for each pairing of the SSRC of command source and the SSRC of the command target. The sequence number SHALL be - increased by 1 modulo 256 for each new command. A repetition - SHALL NOT increase the sequence number. The initial value is - arbitrary. + increased by 1 for each new command (modulo 256, so the value + after 255 is 0). A repetition SHALL NOT increase the sequence + number. The initial value is arbitrary. C (1 bit) A flag bit indicating whether the "Current Temporal Layer ID (CTID)" and "Current Layer ID (CLID)" fields are present in the FCI. If this bit is 0, the sender of the LRR message is requesting refresh of all layers up to and including the target layer. Payload Type (7 bits) The RTP payload type for which the LRR is being requested. This gives the context in which the target layer index is to be interpreted. - Reserved (RES) (16 bits / 5 bits / 5 bits) All bits SHALL be set to - 0 by the sender and SHALL be ignored on reception. + Reserved (RES) (three separate fields, 16 bits / 5 bits / 5 bits) + All bits SHALL be set to 0 by the sender and SHALL be ignored on + reception. Target Temporal Layer ID (TTID) (3 bits) The temporal ID of the target layer for which the receiver wishes a refresh point. Target Layer ID (TLID) (8 bits) The layer ID of the target spatial or quality layer for which the receiver wishes a refresh point. Its format is dependent on the payload type field. Current Temporal Layer ID (CTID) (3 bits) If C is 1, the ID of the current temporal layer being decoded by the receiver. This @@ -278,20 +283,25 @@ layer. If C is 0, this field SHALL be set to 0 by the sender and SHALL be ignored on reception. When C is 1, TTID MUST NOT be less than CTID, and TLID MUST NOT be less than CLID; at least one of TTID or TLID MUST be greater than CTID or CLID respectively. That is to say, the target layer index MUST be a layer upgrade from the current layer index . A sender MAY request an upgrade in both temporal and spatial/quality layers simultaneously. + A receiver receiving an LRR feedback packet which does not satisfy + the requirements of the previous paragraph, i.e. one where the C bit + is present but TTID is less than CTID or TLID is less than CLID, MUST + discard the request. + Note: the syntax of the TTID, TLID, CTID, and CLID fields match, by design, the TID and LID fields in [I-D.ietf-avtext-framemarking]. 3.2. Semantics Within the common packet header for feedback messages (as defined in section 6.1 of [RFC4585]), the "SSRC of packet sender" field indicates the source of the request, and the "SSRC of media source" is not used and SHALL be set to 0. The SSRCs of the media senders to which the LRR command applies are in the corresponding FCI entries. @@ -300,24 +310,25 @@ Upon reception of LRR, the encoder MUST send a decoder refresh point (see Section 2.1) as soon as possible. The sender MUST respect bandwidth limits provided by the application of congestion control, as described in Section 5 of [RFC5104]. As layer refresh points will often be larger than non-refreshing frames, this may restrict a sender's ability to send a layer refresh point quickly. - LRR MUST NOT be sent as a reaction to picture losses -- it is - RECOMMENDED to use PLI [RFC4585] instead. LRR SHOULD be used only in - situations where not sending a layer refresh point would render the - video unusable for the users. + LRR MUST NOT be sent as a reaction to picture losses due to packet + loss or corruption -- it is RECOMMENDED to use PLI [RFC4585] instead. + LRR SHOULD be used only in situations where there is an explicit + change in decoders' behavior, for example when a receiver will start + decoding a layer which it previously had been discarding. 4. Usage with specific codecs In order for LRR to be used with a scalable codec, the format of the temporal and layer ID fields (for both the target and current layer indices) needs to be specified for that codec's RTP packetization. New RTP packetization specifications for scalable codecs SHOULD define how this is done. (The VP9 payload [I-D.ietf-payload-vp9], for instance, has done so.) If the payload also specifies how it is used with the Frame Marking RTP Header Extension @@ -473,59 +485,62 @@ The LRR message is applicable to all these mechanisms. For MRST and MRMT mechanisms, the "media source" field of the LRR FCI is set to the SSRC of the RTP stream containing the layer indicated by the Current Layer Index (if "C" is 1), or the stream containing the base encoded stream (if "C" is 0). For MRMT, it is sent on the RTP session on which this stream is sent. On receipt, the sender MUST refresh all the layers requested in the stream, simultaneously in decode order. -6. Security Considerations - - All the security considerations of FIR feedback packets [RFC5104] - apply to LRR feedback packets as well. Additionally, media senders - receiving LRR feedback packets MUST validate that the payload types - and layer indices they are receiving are valid for the stream they - are currently sending, and discard the requests if not. - -7. SDP Definitions +6. SDP Definitions Section 7 of [RFC5104] defines SDP procedures for indicating and negotiating support for codec control messages (CCM) in SDP. This document extends this with a new codec control command, "lrr", which indicates support of the Layer Refresh Request (LRR). Figure 9 gives a formal Augmented Backus-Naur Form (ABNF) [RFC5234] showing this grammar extension, extending the grammar defined in [RFC5104]. rtcp-fb-ccm-param =/ SP "lrr" ; Layer Refresh Request Figure 9: Syntax of the "lrr" ccm The Offer-Answer considerations defined in [RFC5104] Section 7.2 apply. +7. Security Considerations + + All the security considerations of FIR feedback packets [RFC5104] + apply to LRR feedback packets as well. Additionally, media senders + receiving LRR feedback packets MUST validate that the payload types + and layer indices they are receiving are valid for the stream they + are currently sending, and discard the requests if not. + 8. IANA Considerations This document defines a new entry to the "Codec Control Messages" subregistry of the "Session Description Protocol (SDP) Parameters" registry, according to the following data: Value name: lrr Long name: Layer Refresh Request Command Usable with: ccm + Mux: IDENTICAL-PER-PT + Reference: RFC XXXX + This document also defines a new entry to the "FMT Values for PSFB Payload Types" subregistry of the "Real-Time Transport Protocol (RTP) Parameters" registry, according to the following data: Name: LRR Long Name: Layer Refresh Request Command Value: TBD