--- 1/draft-ietf-avtext-splicing-for-rtp-01.txt 2011-11-15 10:14:10.658671728 +0100 +++ 2/draft-ietf-avtext-splicing-for-rtp-02.txt 2011-11-15 10:14:10.690722790 +0100 @@ -1,18 +1,18 @@ AVTEXT Working Group J. Xia Internet-Draft Huawei -Intended status: Informational October 20, 2011 -Expires: April 22, 2012 +Intended status: Informational November 15, 2011 +Expires: May 18, 2012 Content Splicing for RTP Sessions - draft-ietf-avtext-splicing-for-rtp-01 + draft-ietf-avtext-splicing-for-rtp-02 Abstract This memo outlines RTP splicing. Splicing is a process that replaces the content of the main multimedia stream with other multimedia content, and delivers the substitutive multimedia content to receiver for a period of time. This memo provides some RTP splicing use cases, then we enumerate a set of requirements and analyze whether an existing RTP level middlebox can meet these requirements, at last we provide concrete guidelines for how the chosen middlebox works to @@ -26,21 +26,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on April 22, 2012. + This Internet-Draft will expire on May 18, 2012. Copyright Notice Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -52,32 +52,32 @@ Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. RTP Splicing Discussion and Requirements . . . . . . . . . . . 5 4. Recommended Solution for RTP Splicing . . . . . . . . . . . . 7 4.1. RTP Processing in RTP Mixer . . . . . . . . . . . . . . . 7 4.2. RTCP Processing in RTP Mixer . . . . . . . . . . . . . . . 9 4.3. Media Clipping Considerations . . . . . . . . . . . . . . 10 - 4.4. Congestion Control Considerations . . . . . . . . . . . . 11 + 4.4. Congestion Control Considerations . . . . . . . . . . . . 10 4.5. Processing Splicing in User Invisibility Case . . . . . . 13 5. Implementation Considerations . . . . . . . . . . . . . . . . 13 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 - 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14 + 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 9. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . . 14 9.1. draft-xia-avtext-splicing-for-rtp-01 . . . . . . . . . . . 14 9.2. draft-xia-avtext-splicing-for-rtp-00 . . . . . . . . . . . 14 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 10.1. Normative References . . . . . . . . . . . . . . . . . . . 15 - 10.2. Informative References . . . . . . . . . . . . . . . . . . 16 + 10.2. Informative References . . . . . . . . . . . . . . . . . . 15 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 16 1. Introduction This document outlines how splicing can be used for RTP sessions. Splicing is a process that replaces the content of the main RTP stream with other multimedia content, and delivers the substitutive content to receiver for a period of time. The substitutive content can be provided for example via another RTP stream or local media file storage. @@ -285,88 +285,75 @@ outgoing packets to prevent user from identifying the difference between main RTP stream and substitutive RTP stream) when mixer is used. 4.1. RTP Processing in RTP Mixer Once mixer has learnt when to do splicing, it must get ready for the coming splicing in advance, e.g., fetches the substitutive content either from local media file storage or via substitutive RTP stream earlier than splicing in point. If the substitutive content comes - from local media file storage, mixer can construct the substitutive - RTP stream using its own SSRC and leave the CSRC list blank in the - output stream. + from local media file storage, mixer should leave the CSRC list blank + in the output stream. - When the main RTP stream begins, mixer terminates the main RTP - stream. Using the main RTP packets, mixer generates the current + Even if splicing does not begin, mixer still needs to receive the + main RTP stream, terminate it and generate a media stream as defined + in RFC3550. Using the main RTP packets, mixer generates the current media stream with its own SSRC, sequence number space and timing model. Moreover, mixer inserts the SSRC of main RTP stream into CSRC list in the current media stream. When splicing begins, mixer chooses the substitutive RTP stream as input stream at splicing in point, extracts the payload data (i.e., substitutive content), encodes substitutive content and outputs it instead of main content in the current media stream. Moreover, mixer inserts the SSRC of substitutive RTP stream into CSRC list in the current media stream. When splicing ends, mixer retrieves the main RTP stream as input stream at splicing out point, extracts the payload data (i.e., main content), encodes main content and outputs it instead of substitutive content in the current media stream. Moreover, mixer inserts the SSRC of main RTP stream into CSRC list in the current media stream. The whole RTP splicing procedure is perhaps best explained by a pseudo code example: - if (main RTP stream begins) { - the main RTP stream is terminated on mixer and main content is - encoded by mixer with its own SSRC identifier; - - the sequence numbers of the current RTP packets which contain main - content are allocated by mixer, until the splicing begins; - - the timestamp of the current RTP packet increments linearly; - - the CSRC list of the current RTP packet indicates SSRC of main RTP - stream; - } - - else if (splicing begins) { + if (splicing begins) { the substitutive RTP stream is terminated on mixer and substitutive content is encoded by mixer with its own SSRC identifier; the sequence numbers of the current RTP packets which contain substitutive content are allocated by mixer and maintain consistent with the sequence numbers of previous current RTP packets, until the splicing end; the timestamp of the current RTP packet increments linearly; the CSRC list of the current RTP packet indicates SSRC of substitutive RTP stream; } - else if (splicing ends) { + + else { the main RTP stream is terminated on mixer and main content is encoded by mixer with its own SSRC identifier; the sequence numbers of the current RTP packets which contain main content are allocated by mixer and maintain consistent with the - sequence numbers of previous current RTP packets, until the next + sequence numbers of previous current RTP packets, until the splicing begins; the timestamp of the current RTP packets increments linearly; the CSRC list the current RTP indicates SSRC of main RTP stream; } - Splicing may occur more than one time during the lifetime of main RTP stream, this means mixer needs to output main content and substitutive content in turn with its own SSRC identifier. From user point of view, the only source of the current stream is mixer wherever the content comes from. Note that, the substitutive content should be outputted in the range of splicing duration. Any gap or overlap between main RTP stream and substitutive RTP stream may induce media clipping at splicing point. More details about preventing media clipping are introduced in @@ -404,20 +391,25 @@ receiver report into two separated receiver reports and send them to their original media sources respectively. For each separated receiver report, mixer also needs to make the corresponding changes to the packet loss fields in report block besides the SSRC field and the extended highest sequence number field. Based on above RTCP operating mechanism, the media source will see the reception quality of its stream received by mixer, and the reception quality of spliced stream received by RTP receiver. + If the substitutive content comes from local media file storage ( + i.e., mixer can be regarded as the substitutive media source), the + reception reports should be terminated on mixer without any further + processing. + For the media source whose content is terminated on mixer and is not being passed to receiver, mixer must act as a receiver and send reception reports to the media source. 4.3. Media Clipping Considerations This section provides informative guideline about how media clipping may shape and how mixer deal with the media clipping. If the time slot for substitutive RTP stream mismatches (shorter or @@ -430,31 +422,23 @@ time of substitutive content so that smooth playback can be achieved without pauses or stuttering on RTP receiver. Compared to buffering method used at splicing in point, things become somewhat complex at splicing out point. The case that insertion duration is shorter than the reserved gap time may cause a little playback latency of main RTP stream on RTP receiver, but not adversely impact the quality of user experience. However, in case that insertion duration is longer than the reserved gap duration, there exists an overlap of the substitutive RTP stream and the main - RTP stream at splicing out point, which may cause synchronization - problem and result in a perceptible media clipping. - - To guard against a media clipping at splicing out point, main RTP - source may reserve a bit extra playback delay (e.g., 500 - milliseconds) to send out the first main RTP packet after splicing - ends. Note that the delay should not be too long to smoothly - playback the coming main RTP stream. But if the splicing is still - unfinished when the first main RTP packet has reached, mixer must - terminate the splicing and switch back to main RTP stream even if - this may cause media stuttering on receiver. + RTP stream at splicing out point. In such case, mixer may take a + ungracefule action, terminating the splicing and switching back to + main RTP stream even if this may cause media stuttering on receiver Another reason to cause media clipping is synchronization delay at splicing point if RTP receiver needs to synchronize multiple current streams for playback. How to address this issue is discussed in detail in [RFC6051], which provides three feasible approaches to reduce synchronization delay. 4.4. Congestion Control Considerations Provided that the substitutive content has somewhat different @@ -536,20 +520,25 @@ generating even in the absence of congestion on the path between itself and mixer is desirable. The TMMBR message defined in [RFC5104] provides an effective method. When mixer detects congestion on its downstream link during splicing, it uses TMMBR to request substitutive media source to reduce the media bitrate to a value that is in compliance with congestion control principles for the slowest link. Upon reception of TMMBR, substitutive media source applies its congestion control algorithm and responds Temporary Maximum Media Stream Bit Rate Notification (TMMBN) to mixer. + If the substitutive content comes from local media file storage, + mixer must directly reduce the substitutive media bitrate as the + substitutive media source when it detects any congestion on its + downstream link during splicing. + From above analysis, to reduce the risk of congestion and remain the bandwidth consumption stable over time, the substitutive RTP stream is RECOMMENDED to be encoded at an appropriate bitrate to match that of main RTP stream. If the substitutive RTP stream comes from substitutive media source, the source had better has some knowledge about the media encoding bitrate of main content in advance. How it knows that is out of scope in this draft. 4.5. Processing Splicing in User Invisibility Case