draft-ietf-avtext-splicing-for-rtp-00.txt   draft-ietf-avtext-splicing-for-rtp-01.txt 
AVTEXT Working Group J. Xia AVTEXT Working Group J. Xia
Internet-Draft Huawei Internet-Draft Huawei
Intended status: Informational October 8, 2011 Intended status: Informational October 20, 2011
Expires: April 10, 2012 Expires: April 22, 2012
Content Splicing for RTP Sessions Content Splicing for RTP Sessions
draft-ietf-avtext-splicing-for-rtp-00 draft-ietf-avtext-splicing-for-rtp-01
Abstract Abstract
This memo outlines RTP splicing. Splicing is a process that replaces This memo outlines RTP splicing. Splicing is a process that replaces
the content of the main multimedia stream with other multimedia the content of the main multimedia stream with other multimedia
content, and delivers the substitutive multimedia content to receiver content, and delivers the substitutive multimedia content to receiver
for a period of time. This memo provides some RTP splicing use for a period of time. This memo provides some RTP splicing use
cases, then we enumerate a set of requirements and analyze whether an cases, then we enumerate a set of requirements and analyze whether an
existing RTP level middlebox can meet these requirements, at last we existing RTP level middlebox can meet these requirements, at last we
provide concrete guidelines for how the chosen middlebox works to provide concrete guidelines for how the chosen middlebox works to
skipping to change at page 1, line 37 skipping to change at page 1, line 37
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on February 12, 2012. This Internet-Draft will expire on April 22, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 21 skipping to change at page 2, line 21
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. RTP Splicing Discussion and Requirements . . . . . . . . . . . 5 3. RTP Splicing Discussion and Requirements . . . . . . . . . . . 5
4. Recommended Solution for RTP Splicing . . . . . . . . . . . . 7 4. Recommended Solution for RTP Splicing . . . . . . . . . . . . 7
4.1. RTP Processing in RTP Mixer . . . . . . . . . . . . . . . 7 4.1. RTP Processing in RTP Mixer . . . . . . . . . . . . . . . 7
4.2. RTCP Processing in RTP Mixer . . . . . . . . . . . . . . . 9 4.2. RTCP Processing in RTP Mixer . . . . . . . . . . . . . . . 9
4.3. Media Clipping Considerations . . . . . . . . . . . . . . 10 4.3. Media Clipping Considerations . . . . . . . . . . . . . . 10
4.4. Congestion Control Considerations . . . . . . . . . . . . 11 4.4. Congestion Control Considerations . . . . . . . . . . . . 11
4.5. Processing Splicing in User Invisibility Case . . . . . . 13 4.5. Processing Splicing in User Invisibility Case . . . . . . 13
5. Implementation Considerations . . . . . . . . . . . . . . . . 13 5. Implementation Considerations . . . . . . . . . . . . . . . . 13
6. Security Considerations . . . . . . . . . . . . . . . . . . . 13 6. Security Considerations . . . . . . . . . . . . . . . . . . . 13
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 13 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14
9. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . . 14 9. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1. draft-xia-avtext-splicing-for-rtp-01 . . . . . . . . . . . 14 9.1. draft-xia-avtext-splicing-for-rtp-01 . . . . . . . . . . . 14
9.2. draft-xia-avtext-splicing-for-rtp-00 . . . . . . . . . . . 14 9.2. draft-xia-avtext-splicing-for-rtp-00 . . . . . . . . . . . 14
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15
10.1. Normative References . . . . . . . . . . . . . . . . . . . 15 10.1. Normative References . . . . . . . . . . . . . . . . . . . 15
10.2. Informative References . . . . . . . . . . . . . . . . . . 15 10.2. Informative References . . . . . . . . . . . . . . . . . . 16
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 16 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 16
1. Introduction 1. Introduction
This document outlines how splicing can be used for RTP sessions. This document outlines how splicing can be used for RTP sessions.
Splicing is a process that replaces the content of the main RTP Splicing is a process that replaces the content of the main RTP
stream with other multimedia content, and delivers the substitutive stream with other multimedia content, and delivers the substitutive
content to receiver for a period of time. The substitutive content content to receiver for a period of time. The substitutive content
can be provided for example via another RTP stream or local media can be provided for example via another RTP stream or local media
file storage. file storage.
skipping to change at page 7, line 16 skipping to change at page 7, line 16
requirement as User Invisibility Requirement. requirement as User Invisibility Requirement.
To improve the versatility of existing implementations and better To improve the versatility of existing implementations and better
interoperability, it is RECOMMENDED to use existing tools in RTP/RTCP interoperability, it is RECOMMENDED to use existing tools in RTP/RTCP
protocol family to realize RTP splicing without any protocol protocol family to realize RTP splicing without any protocol
extension unless the existing tools are incompetent for splicing. extension unless the existing tools are incompetent for splicing.
4. Recommended Solution for RTP Splicing 4. Recommended Solution for RTP Splicing
Given that Splicer is an intermediary node exists between the main Given that Splicer is an intermediary node exists between the main
media source and the RTP receiver and splicing is not an very media source and the RTP receiver and splicing is not a very
complicated processing, there are some chance that any existing RTP- complicated processing, there are some chance that any existing RTP-
level middlebox may has the incidental capability to meet the level middlebox may has the incidental capability to meet the
requirements described in previous section. requirements described in previous section.
Since Splicer switches between substitutive content and main content, Since Splicer needs to select substitutive content or main content as
and only forward one of them at one point of time. A RFC3550 mixer the input content at one point of time, an RTP mixer seems to have
seems to have such capability to select a media stream, always under such capability to do this under its own SSRC. Moreover, mixer
its own SSRC, which can be used for reception reports to media includes the CSRC list in outgoing packets to indicate the source(s)
sources and receivers. Moreover, mixer includes the CSRC list in of content, this facilitates the system debugging. From this point
outgoing packets to indicate the source(s) of content, this of view, an RTP mixer may have some chance to be Splicer. In next
facilitates the system debugging. From this point of view, an RTP four subsections (from subsection 4.1 to subsection 4.4), we start
mixer may have some chance to be Splicer. In next four subsections analyzing how an RTP mixer handles RTP splicing and how it satisfies
(from subsection 4.1 to subsection 4.4), we start analyzing how an the general requirements listed in section 3.
RTP mixer handles RTP splicing and how it satisfies the general
requirements listed in section 3.
In subsection 4.5, we specially consider the special requirement 6 In subsection 4.5, we specially consider the special requirement 6
(i.e., User Invisibility Requirement) since it needs to mask any (i.e., User Invisibility Requirement) since it needs to mask any RTP
splicing clue on user (e.g, CSRC list must not be included in splicing clue on user (e.g, CSRC list must not be included in
outgoing packets to prevent user from identifying the difference outgoing packets to prevent user from identifying the difference
between main RTP stream and substitutive RTP stream) when mixer is between main RTP stream and substitutive RTP stream) when mixer is
used. used.
4.1. RTP Processing in RTP Mixer 4.1. RTP Processing in RTP Mixer
Once mixer has learnt when to do splicing, it must get ready for the Once mixer has learnt when to do splicing, it must get ready for the
coming splicing in advance, e.g., fetches the substitutive content coming splicing in advance, e.g., fetches the substitutive content
either from local media file storage or via substitutive RTP stream either from local media file storage or via substitutive RTP stream
skipping to change at page 8, line 9 skipping to change at page 8, line 7
from local media file storage, mixer can construct the substitutive from local media file storage, mixer can construct the substitutive
RTP stream using its own SSRC and leave the CSRC list blank in the RTP stream using its own SSRC and leave the CSRC list blank in the
output stream. output stream.
When the main RTP stream begins, mixer terminates the main RTP When the main RTP stream begins, mixer terminates the main RTP
stream. Using the main RTP packets, mixer generates the current stream. Using the main RTP packets, mixer generates the current
media stream with its own SSRC, sequence number space and timing media stream with its own SSRC, sequence number space and timing
model. Moreover, mixer inserts the SSRC of main RTP stream into CSRC model. Moreover, mixer inserts the SSRC of main RTP stream into CSRC
list in the current media stream. list in the current media stream.
When splicing begins, mixer switches to the substitutive RTP stream When splicing begins, mixer chooses the substitutive RTP stream as
at splicing in point, extracts the payload data (i.e., substitutive input stream at splicing in point, extracts the payload data (i.e.,
content), encodes substitutive content and outputs it instead of main substitutive content), encodes substitutive content and outputs it
content in the current media stream. Moreover, mixer inserts the instead of main content in the current media stream. Moreover, mixer
SSRC of substitutive RTP stream into CSRC list in the current media inserts the SSRC of substitutive RTP stream into CSRC list in the
stream. current media stream.
When splicing ends, mixer switches to the main RTP stream at splicing When splicing ends, mixer retrieves the main RTP stream as input
out point, extracts the payload data (i.e., main content), encodes stream at splicing out point, extracts the payload data (i.e., main
main content and outputs it instead of substitutive content in the content), encodes main content and outputs it instead of substitutive
current media stream. Moreover, mixer inserts the SSRC of main RTP content in the current media stream. Moreover, mixer inserts the
stream into CSRC list in the current media stream. SSRC of main RTP stream into CSRC list in the current media stream.
The whole RTP splicing procedure is perhaps best explained by a The whole RTP splicing procedure is perhaps best explained by a
pseudo code example: pseudo code example:
if (main RTP stream begins) { if (main RTP stream begins) {
the main RTP stream is terminated on mixer and main content is the main RTP stream is terminated on mixer and main content is
encoded by mixer with its own SSRC identifier; encoded by mixer with its own SSRC identifier;
the sequence numbers of the current RTP packets which contain main the sequence numbers of the current RTP packets which contain main
content are allocated by mixer, until the splicing begins; content are allocated by mixer, until the splicing begins;
skipping to change at page 9, line 48 skipping to change at page 9, line 45
information to media source via RTCP reception reports. information to media source via RTCP reception reports.
According to the description in section 7.3 of [RFC3550], mixer According to the description in section 7.3 of [RFC3550], mixer
divides RTCP flow between media source and receiver into two separate divides RTCP flow between media source and receiver into two separate
RTCP loops, media source probably has no idea about the situation on RTCP loops, media source probably has no idea about the situation on
receiver. Hence, mixer may use some mechanisms, allowing media receiver. Hence, mixer may use some mechanisms, allowing media
source to at least some degree to have some knowledge of the source to at least some degree to have some knowledge of the
situation on receiver when its content is being passed to receiver. situation on receiver when its content is being passed to receiver.
Because splicing is a processing that mixer selects one media stream Because splicing is a processing that mixer selects one media stream
from multiple streams rather than mixing them, the number of the from multiple streams rather than mixing them, the number of output
original, pre-spliced RTP packets will be equal to the number of RTP packets containing substitutive content is equal to the number of
spliced RTP packets, i.e., mixer changes the sequence numbers of RTP input substitutive RTP packets (from substitutive RTP stream) during
packets but not changes the sum of them. When mixer receives RTCP splicing, the mixer does not need to modify loss packet fields in
reception reports sent from downstream receiver, it must change the receiver report blocks unless the reporting intervals spans the
SSRC in report block to the SSRC identifier of original media source splicing point. But mixer needs to change the SSRC field in report
and rewrite the extended highest sequence number to the corresponding block to the SSRC identifier of original media source and rewrite the
original extended highest sequence number before forwarding the RTCP extended highest sequence number field to the corresponding original
reception reports to original media source. In such case, the media extended highest sequence number before forwarding the RTCP reception
source will see the reception quality of its stream received by reports to original media source.
mixer, and the reception quality of spliced stream received by RTP
receiver. When a RTCP receiver report spans the splicing point, it reflects the
characteristics of the combination of main RTP packets and
substitutive RTP packets, in which case, mixer needs to divide the
receiver report into two separated receiver reports and send them to
their original media sources respectively. For each separated
receiver report, mixer also needs to make the corresponding changes
to the packet loss fields in report block besides the SSRC field and
the extended highest sequence number field.
Based on above RTCP operating mechanism, the media source will see
the reception quality of its stream received by mixer, and the
reception quality of spliced stream received by RTP receiver.
For the media source whose content is terminated on mixer and is not For the media source whose content is terminated on mixer and is not
being passed to receiver, mixer must act as a receiver and therefore being passed to receiver, mixer must act as a receiver and send
send reception reports to the media source. reception reports to the media source.
4.3. Media Clipping Considerations 4.3. Media Clipping Considerations
This section provides informative guideline about how media clipping This section provides informative guideline about how media clipping
may shape and how mixer deal with the media clipping. may shape and how mixer deal with the media clipping.
Media clippings potentially occur at the splicing point if the time If the time slot for substitutive RTP stream mismatches (shorter or
slot for substitutive RTP stream mismatches (shorter or longer than) longer than) the duration of the reserved main RTP stream for
the duration of the reserved main RTP stream for replacing. replacing, the media clipping may occur at the splicing point which
usually is the joint between two independently decodable frames.
At the splicing in point, mixer usually start the substitutive At the splicing in point, mixer can fill the substitutive content up
content with independently decodable frames and fill the substitutive receiver's buffer with several seconds earlier than the presentation
content up receiver's buffer with several seconds earlier than the time of substitutive content so that smooth playback can be achieved
presentation time of substitutive content. By this way, smooth without pauses or stuttering on RTP receiver.
playback can be achieved without pauses or stuttering.
Compared to buffering method used at splicing in point, things become Compared to buffering method used at splicing in point, things become
somewhat complex at splicing out point. The case that insertion somewhat complex at splicing out point. The case that insertion
duration is shorter than the reserved gap time may cause a little duration is shorter than the reserved gap time may cause a little
playback latency of main RTP stream on RTP receiver, but not playback latency of main RTP stream on RTP receiver, but not
adversely impact the quality of user experience. However, in case adversely impact the quality of user experience. However, in case
that insertion duration is longer than the reserved gap duration, that insertion duration is longer than the reserved gap duration,
there exists an overlap of the substitutive RTP stream and the main there exists an overlap of the substitutive RTP stream and the main
RTP stream at splicing out point, which may cause synchronization RTP stream at splicing out point, which may cause synchronization
problem and result in a perceptible media clipping. problem and result in a perceptible media clipping.
To guard against a media clipping at splicing out point, main RTP To guard against a media clipping at splicing out point, main RTP
source may reserve a bit extra playback delay (e.g., 500 source may reserve a bit extra playback delay (e.g., 500
milliseconds) to send out the first main RTP packet after splicing milliseconds) to send out the first main RTP packet after splicing
ends. Note that the delay should not be too long to smoothly ends. Note that the delay should not be too long to smoothly
playback the coming main RTP stream. But if the splicing is still playback the coming main RTP stream. But if the splicing is still
unfinished when the first main RTP packet has reached, mixer must unfinished when the first main RTP packet has reached, mixer must
terminate the splicing and switch back to main RTP stream even if terminate the splicing and switch back to main RTP stream even if
this may cause media stuttering on receiver. this may cause media stuttering on receiver.
Another reason to cause media clipping is synchronization delay at
splicing point if RTP receiver needs to synchronize multiple current
streams for playback. How to address this issue is discussed in
detail in [RFC6051], which provides three feasible approaches to
reduce synchronization delay.
4.4. Congestion Control Considerations 4.4. Congestion Control Considerations
Provided that the substitutive content has somewhat different Provided that the substitutive content has somewhat different
characteristics to the main content it replaces (e.g., the more characteristics to the main content it replaces (e.g., the more
dynamic content, the higher bandwidth occupation), or substitutive dynamic content, the higher bandwidth occupation), or substitutive
content may be encoded with different codec and has different content may be encoded with different codec and has different
encoding bitrate, some challenge raise to network capacity and encoding bitrate, some challenge raise to network capacity and
receiver buffer size. A more dynamic content or a higher encoding receiver buffer size. A more dynamic content or a higher encoding
bitrate stream might overload the network and possibly exceed the bitrate stream might overload the network and possibly exceed the
receiver's media consumption rate, which might flood receiver's receiver's media consumption rate, which might flood receiver's
skipping to change at page 15, line 37 skipping to change at page 15, line 51
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
July 2006. July 2006.
[RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman, [RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
"Codec Control Messages in the RTP Audio-Visual Profile "Codec Control Messages in the RTP Audio-Visual Profile
with Feedback (AVPF)", RFC 5104, February 2008. with Feedback (AVPF)", RFC 5104, February 2008.
[RFC5117] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 5117, [RFC5117] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 5117,
January 2008. January 2008.
[RFC6051] Perkins, C. and T. Schierl, "Rapid Synchronisation of RTP
Flows", RFC 6051, November 2010.
[I-D.ietf-avtcore-ecn-for-rtp] [I-D.ietf-avtcore-ecn-for-rtp]
Westerlund, M., "Explicit Congestion Notification (ECN) Westerlund, M., "Explicit Congestion Notification (ECN)
for RTP over UDP", draft-ietf-avtcore-ecn-for-rtp-02 (work for RTP over UDP", draft-ietf-avtcore-ecn-for-rtp-02 (work
in progress), October 2010. in progress), October 2010.
10.2. Informative References 10.2. Informative References
[RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP
Friendly Rate Control (TFRC): Protocol Specification", Friendly Rate Control (TFRC): Protocol Specification",
RFC 5348, September 2008. RFC 5348, September 2008.
 End of changes. 16 change blocks. 
52 lines changed or deleted 70 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/