draft-ietf-avtext-avpf-ccm-layered-03.txt   draft-ietf-avtext-avpf-ccm-layered-04.txt 
Network Working Group S. Wenger Network Working Group S. Wenger
Internet-Draft J. Lennox Internet-Draft J. Lennox
Updates: 5104 (if approved) Vidyo, Inc. Updates: 5104 (if approved) Vidyo, Inc.
Intended status: Standards Track B. Burman Intended status: Standards Track B. Burman
Expires: May 21, 2017 M. Westerlund Expires: July 14, 2017 M. Westerlund
Ericsson Ericsson
November 17, 2016 January 10, 2017
Using Codec Control Messages in the RTP Audio-Visual Profile with Using Codec Control Messages in the RTP Audio-Visual Profile with
Feedback with Layered Codecs Feedback with Layered Codecs
draft-ietf-avtext-avpf-ccm-layered-03 draft-ietf-avtext-avpf-ccm-layered-04
Abstract Abstract
This document updates RFC5104 by fixing a shortcoming in the This document updates RFC5104 by fixing a shortcoming in the
specification language of the Codec Control Message Full Intra specification language of the Codec Control Message Full Intra
Request (FIR) as defined in RFC5104 when using it with layered Request (FIR) as defined in RFC5104 when using it with layered
codecs. In particular, a Decoder Refresh Point needs to be sent by a codecs. In particular, a Decoder Refresh Point needs to be sent by a
media sender when a FIR is received on any layer of the layered media sender when a FIR is received on any layer of the layered
bitstream, regardless on whether those layers are being sent in a bitstream, regardless on whether those layers are being sent in a
single or in multiple RTP flows. The other payload-specific feedback single or in multiple RTP flows. The other payload-specific feedback
skipping to change at page 1, line 43 skipping to change at page 1, line 43
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 21, 2017. This Internet-Draft will expire on July 14, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 3, line 10 skipping to change at page 3, line 10
Since the time those RFCs were developed, layered codecs have gained Since the time those RFCs were developed, layered codecs have gained
in popularity and deployment. Layered codecs use multiple sub- in popularity and deployment. Layered codecs use multiple sub-
bitstreams called layers to represent the content in different bitstreams called layers to represent the content in different
fidelities. Depending on the media codec and its RTP payload format fidelities. Depending on the media codec and its RTP payload format
in use, a number of options exist how to transport those layers in in use, a number of options exist how to transport those layers in
RTP. With reference to A Taxonomy of Semantics and Mechanisms for RTP. With reference to A Taxonomy of Semantics and Mechanisms for
Real-Time Transport Protocol (RTP) Sources [RFC7656]): Real-Time Transport Protocol (RTP) Sources [RFC7656]):
single layers or groups of layers may be sent in their own RTP single layers or groups of layers may be sent in their own RTP
streams in MRST or MRMT mode; streams in Multiple RTP streams on a Single media Transport (MRST)
or Multiple RTP streams on Multiple media Transports (MRMT) mode;
using media-codec specific multiplexing mechanisms, multiple using media-codec specific multiplexing mechanisms, multiple
layers may be sent in a single RTP stream in SRST mode. layers may be sent in a single RTP stream in Single RTP stream on
a Single media Transport (SRST) mode.
The dependency relationship between layers in a truly layered, The dependency relationship between layers in a truly layered,
pyramid-shaped bitstream forms a directed graph, with the base layer pyramid-shaped bitstream forms a directed graph, with the base layer
at the root. Enhancement layers depend on the base layer and at the root. Enhancement layers depend on the base layer and
potentially on other enhancement layers, and the target layer and all potentially on other enhancement layers, and the target layer and all
layers it depends on have to be decoded jointly in order to re-create layers it depends on have to be decoded jointly in order to re-create
the uncompressed media signal at the fidelity of the target layer. the uncompressed media signal at the fidelity of the target layer.
Such a layering structure is assumed henceforth; for more exotic Such a layering structure is assumed henceforth; for more exotic
layering structures please see Section 5. layering structures please see Section 5.
Implementation experience has shown that the Full Intra Request Implementation experience has shown that the Full Intra Request (FIR)
command as defined in [RFC5104] is underspecified when used with command as defined in [RFC5104] is underspecified when used with
layered codecs and when more than one RTP stream is used to transport layered codecs and when more than one RTP stream is used to transport
the layers of a layered bitstream at a given fidelity. In the layers of a layered bitstream at a given fidelity. In
particular, from the [RFC5104] specification language it is not clear particular, from the [RFC5104] specification language it is not clear
whether an FIR received for only a single RTP stream of multiple RTP whether an FIR received for only a single RTP stream of multiple RTP
streams covering the same layered bitstream necessarily triggers the streams covering the same layered bitstream necessarily triggers the
sending of a Decoder Refresh Point (as defined in [RFC5104] section sending of a Decoder Refresh Point (as defined in [RFC5104] section
2.2) for all layers, or only for the layer which is transported in 2.2) for all layers, or only for the layer which is transported in
the RTP stream which the FIR request is associated with. the RTP stream that the FIR request is associated with.
This document fixes this shortcoming by: This document fixes this shortcoming by:
a. Updating the definition of the Decoder Refresh Point (as defined a. Updating the definition of the Decoder Refresh Point (as defined
in [RFC5104] section 2.2) to cover layered codecs, in line with in [RFC5104] section 2.2) to cover layered codecs, in line with
the corresponding definitions used in a popular layered codec the corresponding definitions used in a popular layered codec
format, namely H.264/SVC [H.264]. Specifically, a decoder format, namely H.264/SVC [H.264]. Specifically, a decoder
refresh point, in conjunction with layered codecs, resets the refresh point, in conjunction with layered codecs, resets the
state of the whole decoder, which implies that it includes hard state of the whole decoder, which implies that it includes hard
or gradual single-layer decoder refresh for all layers; or gradual single-layer decoder refresh for all layers;
b. Require a media sender to send a Decoder Refresh Point after the b. Require a media sender to send a Decoder Refresh Point after the
media sender has received a Full Intra Request over an RTCP media sender has received a FIR over an RTCP stream associated
stream associated with any of the RTP streams over which a part with any of the RTP streams over which a part of the layered
of the layered bitstream is transported; bitstream is transported;
c. Require that a media receiver sends the FIR on the RTCP stream c. Require that a media receiver sends the FIR on the RTCP stream
associated with the base layer. The option of receiving FIR on associated with the base layer. The option of receiving FIR on
enhancement layer-associated RTCP stream as specified in point b) enhancement layer-associated RTCP stream as specified in point b)
above is kept for backward compatibility; and above is kept for backward compatibility; and
d. Providing guidance on how to detect that a layered bitstream is d. Providing guidance on how to detect that a layered bitstream is
in use for which the above rules apply. in use for which the above rules apply.
While, clearly, the reaction to FIR for layered codecs in [RFC5104] While, clearly, the reaction to FIR for layered codecs in [RFC5104]
skipping to change at page 6, line 21 skipping to change at page 6, line 23
bitstream (as defined in the video coding standard), but without bitstream (as defined in the video coding standard), but without
inter-layer prediction. In such a scenario, it is potentially, inter-layer prediction. In such a scenario, it is potentially,
though not necessarily, counter-productive to send a decoder refresh though not necessarily, counter-productive to send a decoder refresh
point on all RTP streams using that payload format and SSRC. It is point on all RTP streams using that payload format and SSRC. It is
beyond the scope of this document to discuss optimized reactions to beyond the scope of this document to discuss optimized reactions to
FIRs received on RTP streams carrying such exotic bitstreams. FIRs received on RTP streams carrying such exotic bitstreams.
One good indication of the likely use of pyramid-shaped layering with One good indication of the likely use of pyramid-shaped layering with
interlayer prediction is when the various RTP streams are "bound" interlayer prediction is when the various RTP streams are "bound"
together on the signaling level. In an SDP environment, this would together on the signaling level. In an SDP environment, this would
be the case if they are marked as being dependent from each other be the case if they are marked as being dependent on each other using
using The Session Description Protocol (SDP) Grouping Framework The Session Description Protocol (SDP) Grouping Framework [RFC5888]
[RFC5888] and the layer dependency RFC 5583 [RFC5583]. and the layer dependency RFC 5583 [RFC5583].
6. Layered Codecs and non-FIR codec control messages (Informative) 6. Layered Codecs and non-FIR codec control messages (Informative)
Between them, AVPF [RFC4585] and Codec Control Messages [RFC5104] Between them, AVPF [RFC4585] and Codec Control Messages [RFC5104]
define a total of seven Payload-specific Feedback messages. For the define a total of seven Payload-specific Feedback messages. For the
FIR command message, guidance has been provided above. In this FIR command message, guidance has been provided above. In this
section, some information is provided with respect to the remaining section, some information is provided with respect to the remaining
six codec control messages. six codec control messages.
6.1. Picture Loss Indication (PLI) 6.1. Picture Loss Indication (PLI)
skipping to change at page 6, line 47 skipping to change at page 6, line 49
that enhancement layer and all dependent enhancement layers through that enhancement layer and all dependent enhancement layers through
appropriate source-coding specific means. However, the reference appropriate source-coding specific means. However, the reference
layer(s) used by the enhancement layer for which the PLI was received layer(s) used by the enhancement layer for which the PLI was received
does not require repair. The encoder can figure out by itself what does not require repair. The encoder can figure out by itself what
constitutes a dependent enhancement layer and does not need help from constitutes a dependent enhancement layer and does not need help from
the system stack in doing so. Thus, there is nothing that needs to the system stack in doing so. Thus, there is nothing that needs to
be specified herein. be specified herein.
6.2. Slice Loss Indication (SLI) 6.2. Slice Loss Indication (SLI)
SLI is defined in section 6.3.2 of [RFC4585]. The authors' current SLI is defined in section 6.3.2 of [RFC4585]. The current
understanding is that the prudent response to a SLI message received understanding is that the prudent response to a SLI message received
for an enhancement layer is to "repair" the affected spatial area of for an enhancement layer is to "repair" the affected spatial area of
that enhancement layer and all dependent enhancement layers through that enhancement layer and all dependent enhancement layers through
appropriate source-coding specific means. As in PLI, the reference appropriate source-coding specific means. As in PLI, the reference
layers used by the enhancement layer for which the SLI was received layers used by the enhancement layer for which the SLI was received
do not need to be repaired. Again, as in PLI, the encoder can do not need to be repaired. Again, as in PLI, the encoder can
determine by itself what constitutes a dependent enhancement layer determine by itself what constitutes a dependent enhancement layer
and does not need help from the system stack in doing so. Thus, and does not need help from the system stack in doing so. Thus,
there is nothing that needs to be specified herein. SLI has seen there is nothing that needs to be specified herein. SLI has seen
very little implementation and, as far as it is known, none in very little implementation and, as far as it is known, none in
conjunction with layered systems. conjunction with layered systems.
6.3. Reference Picture Selection Indication (RPSI) 6.3. Reference Picture Selection Indication (RPSI)
RPSI is defined in section 6.3.3 of [RFC4585]. While a technical RPSI is defined in section 6.3.3 of [RFC4585]. While a technical
equivalent of RPSI has been in use with non-layered systems for many equivalent of RPSI has been in use with non-layered systems for many
years, no implementations are known in conjunction of layered codecs. years, no implementations are known in conjunction of layered codecs.
The authors' current understanding is that the reception of an RPSI The current understanding is that the reception of an RPSI message on
message on any layer indicating a missing reference picture forces any layer indicating a missing reference picture forces the encoder
the encoder to appropriately handle that missing reference picture in to appropriately handle that missing reference picture in the layer
the layer indicated, and all dependent layers. Thus, RPSI should indicated, and all dependent layers. Thus, RPSI should work without
work without further need for specification language. further need for specification language.
6.4. Temporal-Spatial Trade-off Request and Notification (TSTR/TSTN) 6.4. Temporal-Spatial Trade-off Request and Notification (TSTR/TSTN)
TSTN/TSTR are defined in section 4.3.2 and 4.3.3 of [RFC5104], TSTN/TSTR are defined in section 4.3.2 and 4.3.3 of [RFC5104],
respectively. The TSTR request communicates guidance of the respectively. The TSTR request communicates guidance of the
preferred trade-off between spatial quality and frame rate. A preferred trade-off between spatial quality and frame rate. A
technical equivalent of TSTN/TSTR has seen deployment for many years technical equivalent of TSTN/TSTR has seen deployment for many years
in non-scalable systems. in non-scalable systems.
The Temporal-Spatial Trade-off request and notification messages The Temporal-Spatial Trade-off request and notification messages
include an SSRC target, which, similarly to FIR, may refer to an RTP include an SSRC target, which, similarly to FIR, may refer to an RTP
stream carrying a base layer, an enhancement layer, or multiple stream carrying a base layer, an enhancement layer, or multiple
layers. Therefore, the authors' current understanding is that the layers. Therefore, the current understanding is that the semantics
semantics of the message applies to the layers present in the of the message applies to the layers present in the targeted RTP
targeted RTP stream. stream.
It is noted that per-layer TSTR/TSTN is a mechanism that is, in some It is noted that per-layer TSTR/TSTN is a mechanism that is, in some
ways, counterproductive in a system using layered codecs. Given a ways, counterproductive in a system using layered codecs. Given a
sufficiently complex layered bitstream layout, a sending system has sufficiently complex layered bitstream layout, a sending system has
flexibility in adjusting the spatio/temporal quality balance by flexibility in adjusting the spatio/temporal quality balance by
adding and removing temporal, spatial, or quality enhancement layers. adding and removing temporal, spatial, or quality enhancement layers.
At present it is unclear whether an allowed (or even recommended) At present it is unclear whether an allowed (or even recommended)
option to the reception of a TSTR is to adjust the bit allocation option to the reception of a TSTR is to adjust the bit allocation
within the layer(s) present in the addressed RTP stream, or to adjust within the layer(s) present in the addressed RTP stream, or to adjust
the layering structure accordingly--which can involve more than just the layering structure accordingly--which can involve more than just
 End of changes. 14 change blocks. 
24 lines changed or deleted 26 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/