wdiff draft-ietf-avtext-splicing-for-rtp-07.txt draft-ietf-avtext-splicing-for-rtp-08.txt

AVTEXT Working Group J. Xia
Internet-Draft Huawei
Intended status: Informational February 20, July 9, 2012
Expires: August 23, 2012 January 10, 2013

Content Splicing for RTP Sessions
draft-ietf-avtext-splicing-for-rtp-07
draft-ietf-avtext-splicing-for-rtp-08

Abstract

This memo outlines RTP splicing. Splicing

Content splicing is a process that replaces the content of the a main
multimedia stream with other multimedia content, and delivers the
substitutive multimedia content to receiver the receivers for a period of
time. Splicing is commonly used for local advertisement insertion by
cable operators, replacing a national advertisement content with a
local advertisement.

This memo provides describes some RTP splicing use
cases, then we enumerate cases for content splicing and a set of
requirements and analyze whether an
existing RTP level middlebox can meet these requirements, at last we
provide for splicing content delivered by RTP. It provides
concrete guidelines for how the chosen middlebox works a RTP mixer can be used to handle RTP content
splicing.

Status of this Memo

This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."

This Internet-Draft will expire on August 23, 2012. January 10, 2013.

This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.

Table of Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. System Model and Terminology . . . . . . . . . . . . . . . . . 3
3. Requirements for RTP Splicing . . . . . . . . 3
3. RTP Splicing Discussion and Requirements . . . . . . . . . . . 4 6
4. Recommended Solution Content Splicing for RTP Splicing sessions . . . . . . . . . . . . . . 7
4.1. RTP Processing in RTP Mixer . . . . . . . . . . . . . . . 7
4.2. RTCP Processing in RTP Mixer . . . . . . . . . . . . . . . 9 8
4.3. Media Clipping Considerations . . . . . . . . . . . . . . 10
4.4. Congestion Control Considerations . . . . . . . . . . . . 11
4.5. Processing Splicing in User Invisibility Case . . . . . . 13 12
5. Implementation Considerations . . . . . . . . . . . . . . . . 13
6. Security Considerations . . . . . . . . . . . . . . . . . . . 14 13
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 14
9. Change Log . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9.1. draft-xia-avtext-splicing-for-rtp-01 . . . . . . . . 10. Appendix- Why Mixer Is Chosen . . . 14
9.2. draft-xia-avtext-splicing-for-rtp-00 . . . . . . . . . . . 14
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15
10.1. Normative References . . . . . . . . . . . . . . . . . . . 15
10.2. Informative References . . . . . . . . . . . . . . . . . . 16 15
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . . 16

1. Introduction

This document outlines how content splicing can be used for in RTP
sessions.
Splicing Splicing, in general, is a process that replaces the content where part of the main RTP
stream a
multimedia content is replaced with other multimedia content, and delivers the substitutive
content
delivered to receiver the receivers for a period of time. The substitutive
content can be provided for example via another RTP stream or via local
media file storage. One representative use case for splicing is advertisements
local advertisement insertion,
which allows operators allowing content providers to replace
the national advertising content with its own regional advertising
content prior to delivering the regional advertising content to receiver. the
receivers. Besides the advertisement insertion use case, there are
other use cases to in which RTP splicing technology can apply. be applied. For
example, splicing a recorded video into a video conferencing session, and
or implementing a playlist server that stitches pieces of video together
and so forth.

So far
together.

Content splicing is a well-defined operation in MPEG-based cable TV
systems. Indeed, the Society for Cable Telecommunications Engineers
(SCTE) has created two standards, [SCTE30] and [SCTE35] have standardized [SCTE35], to
standardize MPEG2-TS splicing
running over cable. The introduction procedure. SCTE 30 creates a
standardized method for communication between advertisements server
and splicer, and SCTE 35 supports splicing of MPEG2 transport
streams.

When using multimedia splicing into
internet requires changes to transport layer, but to date there is no
guideline for how to handle content splicing for RTP sessions
[RFC3550]. the internet, the media may be
transported by RTP. In this document, we first describe a set of requirements of RTP
splicing. Then we provide a method about how an intermediary node
can be used case the original media content and
substitutive media content will use the same time period, but may
contain different numbers of RTP packets due to process different media
codecs and entropy coding. This mismatch may require some
adjustments of the RTP header sequence number to maintain
consistency. [RFC3550] provides the tools to enabled seamless
content splicing in RTP session, but to meet date there has been no clear
guidelines on how to use these requirements from
the aspects of feasibility, implementation complexity and backward
compatibility.

2. Terminology tools.

This document uses memo outlines the following terminologies.

Current requirements for content splicing in RTP Stream

The
sessions and describes how an RTP stream that mixer can be used to meet these
requirements.

2. System Model and Terminology

In this document, an intermediary network element, the Splicer
handles RTP receiver is currently receiving. splicing. The
content of current RTP stream Splicer can be either receive main content or and
substitutive content.

Main Content

The multimedia content that are conveyed in main RTP stream. Main content simultaneously, but will be replaced by the substitutive content during
splicing.

Main RTP Stream

The send one of them at one
point of time.

When RTP stream that splicing begins, the Splicer is receiving. The sends the substitutive content
to the RTP receiver instead of the main
RTP stream can be replaced by substitutive content for a period of time.

Substitutive Content

The multimedia
When RTP splicing ends, the Splicer switches back sending the main
content that replaces to the RTP receiver.

A simplified RTP splicing diagram is depicted in Figure 1, in which
only one main content during
splicing. The flow and one substitutive content flow are
given. Actually, the Splicer can handle multiple splicing for example be contained
in an RTP stream from a media sender or fetched from local media
file storage.

Substitutive
multiple RTP Stream

A sessions simultaneously. RTP stream that splicing may provide substitutive content. Substitutive
RTP stream and main RTP stream are two separate streams. If the
substitutive content is provided via substitutive RTP stream, the
substitutive RTP Stream must pass through Splicer before the
substitutive content is delivered to receiver.

Splicing In Point

A virtual point in the RTP stream, suitable for substitutive
content entry, that exists happen more
than once in multiple time slots during the boundary lifetime of two independently
decodable frames.

Splicing Out Point

A virtual point in the RTP stream, suitable for substitutive
content exit, that exists in the boundary of two independently
decodable frames.

Splicer

An intermediary node that inserts substitutive content into main RTP
stream. The methods how Splicer sends substitutive content learns when to RTP receiver
instead of main content during splicing. It is also responsible
for processing RTCP traffic between media source and RTP receiver.

3. RTP Splicing Discussion start and Requirements

In this document, we assume an intermediary network element, which is
referred to as Splicer, to play end the key role to handle RTP splicing.
A simplified RTP
splicing diagram is depicted in Figure 1, in which
only one main content flow and one substitutive content flow are
given. out of scope for this document.

+---------------+
| | Main Content +-----------+
|Main
| Main RTP Sender|------------->| |------------->| | Current Output Content
| Content | | Splicer |----------> |--------------->
+---------------+ ---------->| |
| +-----------+
|
| Substitutive Content
|
|
+-----------------------+
|Substitutive
| Substitutive RTP Sender| |
| Content |
| or |
| Local File Storage |
+-----------------------+

Figure 1: RTP Splicing Architecture

When RTP splicing begins, Splicer stops delivering

This document uses the main content,
instead delivering following terminologies.

Output RTP Stream

The RTP stream that the substitutive content to RTP receiver for a
period is currently receiving. The
content of time, and then resumes the current RTP stream can be either main content when splicing ends. or
substitutive content.

Main Content

The methods how Splicer learns when to start and end the splicing is
out of scope for this document. The RTP splicing may happen more
than once in case multimedia content that substitutive are conveyed in main RTP stream. Main
content will be dispersedly
inserted in multiple time slots during the lifetime of replaced by the main substitutive content during
splicing.

Main RTP
stream.

When realizing splicing technology on Stream

The RTP layer, there are a set of
requirements stream that must be satisfied to at least some degree on
Splicer:

REQ-1:

Splicer must operate in either unicast or multicast session
environment.

REQ-2:

Splicer should not cause perceptible media clipping at the
splicing point and adverse impact on the quality of user
experience.

REQ-3:

Splicer must be backward compatible with RTP/RTCP protocols, and
its associated profiles and extensions to those protocols. For
example, Splicer must be robust to packet loss, network congestion
etc.

REQ-4: Splicer must is receiving. The content of main
RTP stream can be trusted replaced by media source and receiver, and has the
valid security context with media source and RTP receiver
respectively.

REQ-5:

Splicer should allow the media source to learn the performance of
the downstream receiver when its substitutive content is being passed to RTP
receiver.

In for a number period of deployment scenarios, especially advertisement
insertion, there may be one specific requirement. Given that it is
unacceptable for advertisers that their advertising content is not
delivered to user, this may require
time.

Main RTP splicing to be operated
within Sender

The sender of RTP packets carrying the following constraint:

REQ-6:

If Splicer intends to prevent main RTP receiver from identifying and
filtering stream.

Substitutive Content

The multimedia content that replaces the main content during
splicing. The substitutive content, it should eliminate the
visibility of splicing process on content can for example be contained
in an RTP level stream from a media sender or fetched from local media
file storage.

Substitutive RTP receiver
point of view.

However, substitutive Stream

The multimedia content and that replaces the main content are encoded by
different encoders and have different parameter sets. In such
case, a full media transcoding must during
splicing. The substitutive content can for example be done on Splicer to ensure
the completely invisible impact on contained
in an RTP receiver, but this may be
prohibitively expensive and complex. As stream from a trade-off, it is
recommended to minimize the splicing visibility on media sender or fetched from local media
file storage.

Substitutive RTP receiver,
i.e., maintaining Sender

The sender of RTP header parameters consistent but leaving packets carrying the substitutive RTP payload untranscoded. If one wants to realize complete
invisibility, the cost of transcoding must be taken into account.

Henceforth, we refer to stream.

Splicing In Point

A virtual point in the minimum and complete invisibility
requirement as User Invisibility Requirement.

To improve RTP stream, suitable for substitutive
content entry, typically in the versatility of existing implementations and better
interoperability, it is recommended to use existing tools boundary between two independently
decodable frames.

Splicing Out Point

A virtual point in RTP/RTCP
protocol family to realize RTP splicing without any protocol
extension unless the existing tools are incompetent for splicing.

4. Recommended Solution for RTP Splicing

Given that stream, suitable for substitutive
content exist, typically in the boundary between two independently
decodable frames.

Splicer is an

An intermediary node exists between the that inserts substitutive content into main
media source and the
RTP receiver and splicing is not a very
complicated processing, there are some chance that any existing RTP-
level middlebox may has the incidental capability to meet the
requirements described in previous section.

Since stream. The Splicer needs to select sends substitutive content or to RTP
receiver instead of main content as during splicing. It is also
responsible for processing RTCP traffic between the input content at one point of time, an RTP mixer seems to have
such capability to do this under its own SSRC. Moreover, mixer may
include sender and
the CSRC list in outgoing packets RTP receiver.

3. Requirements for RTP Splicing

In order to indicate the source(s)
of allow seamless content in some use cases like conferencing, this facilitates splicing at the
system debugging and loop detection. From this point of view, an RTP
mixer may have some chance to layer, the
following requirements must be Splicer. In next four subsections
(from subsection 4.1 to subsection 4.4), we start analyzing how an
RTP mixer handles RTP met. Meeting these will also allow,
but not require, seamless content splicing at layers above RTP.

REQ-1:

The splicer should be agnostic about the network and how it satisfies transport
layer protocols used to deliver the general
requirements listed in section 3.

In subsection 4.5, we specially consider RTP streams.

REQ-2:

The splicing operation at the special requirement 6
(i.e., User Invisibility Requirement) since it needs to mask any RTP layer must allow splicing clue on receiver (e.g, CSRC list at any
point required by the media content, and must not be included constrain when
splicing in
outgoing packets to prevent receiver from identifying the difference
between main or splicing out operations can take place.

REQ-3:

Splicing of RTP stream content must be backward compatible with the RTP/
RTCP protocol, associated profiles, payload formats, and substitutive
extensions.

REQ-4:

A content splicer will modify the content of RTP stream) when mixer packets, and
break the end-to-end security, e.g., breaking data integrity and
source authentication. If the Splicer is
used.

4.1. RTP Processing in RTP Mixer

Once mixer has learnt when designated to do splicing, insert
substitutive content, it must get ready for the
coming splicing be trusted, i.e., be in advance, e.g., fetches the substitutive content
either from local media file storage or via substitutive same
security context as the main RTP stream
earlier than splicing in point. If sender, the substitutive content comes
from local media file storage, mixer should leave RTP
sender, and the CSRC list blank
in receivers. If encryption is employed, the output stream.

Even if splicing does not begin, mixer still needs Splicer
must be able to receive decrypt the
main inbound RTP stream, packets and generate a media stream as defined in RFC3550.
Using main content, mixer generates re-encrypt the current media stream with its
own SSRC, sequence number space
outbound RTP packets after splicing.

REQ-5:

The splicer should rewrite as necessary and timing model. Moreover, mixer
may insert forward RTCP messages
(e.g., including packet loss, jitter, etc.) sent from downstream
receiver to the SSRC of main RTP stream into CSRC list in the current
media stream.

When splicing begins, mixer chooses sender or the substitutive RTP stream as
input stream at splicing in point, sender,
and extracts thus allow the payload data
(i.e., substitutive content). After that, mixer encapsulates
substitutive content instead of main content as RTP sender or substitutive RTP sender to
learn the payload performance of the
current media stream, and then outputs the current media stream downstream receiver when its content
is being passed to RTP receiver. Moreover, mixer may insert In addition, the SSRC of splicer should
rewrite RTCP messages from the main RTP sender or substitutive RTP
stream into CSRC list in
sender to the current media stream.

When splicing ends, mixer retrieves receiver.

REQ-6:

The splicer must not affect other RTP sessions running between the main
RTP stream as input
stream at splicing out point, sender and extracts the payload data (i.e.,
main content). After that, mixer encapsulates main content instead
of substitutive content as the payload of the current media stream, RTP receiver, and then outputs must be transparent for the current media stream
RTP sessions it does not splice.

REQ-7:

The content splicer should be able to receiver. Moreover,
mixer may insert modify the SSRC of main RTP stream into CSRC list across
a splicing in or splicing out point such that the
current media stream.

The whole RTP splicing procedure point
is perhaps best explained by a
pseudo code example:

if (splicing begins) { not easy to detect in the substitutive RTP stream is terminated on mixer and
substitutive content stream. For the advertisement
insertion use case, it is encapsulated by mixer with its own SSRC
identifier; important to make it difficult for the sequence numbers of
receiver to detect it. Ensuring the current RTP packets which contain
substitutive splicing point is not visible
in the media content are allocated by mixer and maintain
consistent may be easy with some codecs, but extremely
difficult with others; in the sequence numbers of previous current RTP
packets, until worst case, the splicer may need to
perform full media transcoding if it has to hide the splicing end;
point in the timestamp of media content. This memo only focusses on making the
splicing invisible at the current RTP packet increments linearly; layer. How (or if) the CSRC list of splicing is
made invisible in the current media stream is outside the scope of this
memo.

4. Content Splicing for RTP packet may include SSRC sessions

The RTP specification [RFC3550] defines two types of
substitutive middlebox: RTP
translators and RTP mixers. Splicing is best viewed as a mixing
operation. The splicer generates a new RTP stream;
}

else { stream that is a mix of
the main RTP stream is terminated on mixer and main content the substitutive RTP stream. An RTP mixer is
encapsulated by
therefore an appropriate model for a content splicer. In next four
subsections (from subsection 4.1 to subsection 4.4), the document
analyzes how the mixer with its own SSRC identifier; handles RTP splicing and how it satisfies the sequence numbers of
general requirements listed in section 3. In subsection 4.5, the current
document looks at REQ-7 in order to hide the fact that splicing take
place.

4.1. RTP packets which contain main Processing in RTP Mixer

A content are allocated by splicer should be implemented as a mixer that receives the
main RTP stream and maintain consistent with the
sequence numbers of previous current substitutive content (possibly via a
substitutive RTP packets, until stream), and sends a single output RTP stream to the
splicing begins;
receiver(s). That output RTP stream will contain either the timestamp of main
content or the current substitutive content. The output RTP packets increments linearly; stream will come
from the CSRC list of mixer, and will have the current RTP may include SSRC of main RTP
stream;
}
Splicing may occur more the mixer rather than one time during the lifetime of
main RTP
stream, this means mixer needs to output main content and sender or the substitutive content in turn with RTP sender.

The mixer uses its own SSRC identifier. From
receiver point of view, SSRC, sequence number space and timing model
when generating the only source of output stream. Moreover, the current stream is mixer wherever the content comes from.

Note that, the substitutive content should be outputted in may insert
the range SSRC of splicing duration. Any gap or overlap between main RTP stream and into CSRC list in the output media
stream.

At the splicing in point, when the substitutive content becomes
active, the mixer chooses the substitutive RTP stream may induce media clipping as input stream
at splicing point.
More details about preventing media clipping are introduced in
section 4.3.

4.2. RTCP Processing in RTP Mixer

By monitoring available bandwidth and buffer levels and by computing
network metrics such as packet loss, network jitter, point, and delay, RTP
receiver can learn extracts the situation on it and can communicate this
information to payload data (i.e.,
substitutive content). If the substitutive content comes from local
media source via RTCP reception reports.

According to file storage, the description in section 7.3 of [RFC3550], mixer
divides RTCP flow between directly fetches the substitutive
content. After that, the mixer encapsulates substitutive content
instead of main content as the payload of the output media source stream,
and receiver into two separate
RTCP loops, media source probably has no idea about then sends the situation on
receiver. Hence, mixer can use some mechanisms, allowing output RTP media
source to at least some degree stream to have some knowledge receiver. The mixer
may insert the SSRC of substitutive RTP stream into CSRC list in the
situation on receiver when its
output media stream. If the substitutive content is being passed to receiver.

Because comes from local
media file storage, the mixer should leave the CSRC list blank.

At the splicing is a processing that out point, when the substitutive content ends, the
mixer selects one media retrieves the main RTP stream
from multiple streams but neither mixing nor transcoding them, upon
receiving an RTCP receiver report from downstream receiver, mixer can
forward it to original media source with its SSRC identifier intact as input stream at splicing out
point, and extracts the payload data (i.e., main content). After
that, the SSRC mixer encapsulates main content instead of downstream receiver). Given that substitutive
content as the number payload of the output RTP packets containing substitutive content is equal media stream, and then sends the
output media stream to the
number of input substitutive RTP packets (from substitutive RTP
stream) during splicing. In receivers. Moreover, the same manner, mixer may insert
the number SSRC of output
RTP packets containing main RTP stream into CSRC list in the output media stream
as before.

Note that if the content is equal too large to the number of input
main fit into RTP packets (from main sent to
RTP stream) during non-splicing, so receiver, the mixer needs to transcode or perform application-
layer fragmentation. Usually the mixer is deployed as part of a
managed system and MTU will be carefully managed by this system.
This document does not need raise any new MTU related issues compared to modify loss packet fields a
standard mixer described in Receiver Report Blocks
unless [RFC3550].

Splicing may occur more than once during the reporting intervals spans lifetime of main RTP
stream, this means the splicing point. But mixer needs to change the SSRC field send main content and
substitutive content in report block to the turn with its own SSRC identifier identifier. From
receiver point of original media source and rewrite view, the extended highest sequence
number field to the corresponding original extended highest sequence
number before forwarding the RTCP receiver report to original media
source.

When a RTCP receiver report spans only source of the splicing point, it reflects output stream is the
characteristics
mixer regardless of where the combination of main content is coming from.

4.2. RTCP Processing in RTP packets Mixer

By monitoring available bandwidth and
substitutive buffer levels and by computing
network metrics such as packet loss, network jitter, and delay, RTP packets, in which case, mixer needs to divide the
receiver report into two separated
receiver reports can learn the network performance and send them to
their original media sources respectively. For each separated
receiver report, mixer also needs communicate this to make
the corresponding changes RTP sender via RTCP reception reports.

According to the packet loss fields description in report block besides the SSRC field and section 7.3 of [RFC3550], the extended highest sequence number field.

The mixer can also inform
splits the media source of quality with which RTCP flow between sender and receiver into two separate
RTCP loops, RTP sender has no idea about the
content reaches situation on the mixer. This
receiver. But splicing is done by a processing that the mixer generating RTCP
reports for selects one
media stream from multiple streams rather than mixing them, so the RTP stream, which it sends upstream towards
mixer can leave the SSRC identifier in the media
source. These RTCP reports use report intact (i.e.,
the SSRC of downstream receiver), this enables the mixer.

Based on above RTCP operating mechanism, main RTP sender or
the media source whose
content is being passed substitutive RTP sender to receiver, will see learn the reception quality
of its stream received situation on mixer, and the reception quality of spliced
stream received on receiver. The media source whose content is not
being passed

When the RTCP report corresponds to receiver, will only see a time interval that is entirely
main content or entirely substitutive content, the reception quality number of its
stream received on mixer.

If the output
RTP packets containing substitutive content comes from local media file storage (
i.e., mixer can be regarded as is equal to the number of
input substitutive media source), RTP packets (from substitutive RTP stream) during
splicing, in the
reception reports received from downstream relate to same manner, the substitutive number of output RTP packets
containing main content should be terminated on is equal to the number of input main RTP
packets (from main RTP stream) during non-splicing unless the mixer without any further processing.

4.3. Media Clipping Considerations
fragment the input RTP packets. This section provides informative guideline about how media clipping
may shape and how means that the mixer deal with does not
need to modify the media clipping.

If loss packet fields in reception report blocks in
RTCP reports. But if the time slot mixer fragments the input RTP packets, it
may need to modify the loss packet fields to compensate for substitutive the
fragmentation. Whether the input RTP stream mismatches (shorter packets are fragmented or
longer than) not,
the duration mixer still needs to change the SSRC field in report block to the
SSRC identifier of the reserved main RTP stream for
replacing, sender or the media clipping may occur at substitutive RTP
sender, and rewrite the splicing point which
usually is extended highest sequence number field to the joint between two independently decodable frames.

At
corresponding original extended highest sequence number before
forwarding the RTCP report to the main RTP sender or the substitutive
RTP sender.

When the RTCP report spans the splicing in point, mixer can fill up receiver's buffer with
substitutive content several seconds earlier than the presentation
time of substitutive content so that smooth playback can be achieved
without pauses point or stuttering on RTP receiver.

Compared to buffering method used at splicing in point, things become
somewhat complex at the splicing out point. The case that insertion
duration is shorter than
point, it reflects the reserved gap time may cause a little
playback latency characteristics of the combination of main RTP stream on
packets and substitutive RTP receiver, but not
adversely impact packets. In this case, the quality of user experience. One alternative
approach is that mixer may pad some blank content (e.g., all black
sequence) needs
to fill up divide the gap. Another alternative approach is that
main media source may RTCP report into two separate RTCP reports and send filler content (e.g., static channel
identifier) during splicing,
them to their original RTP senders respectively. For each RTCP
report, the mixer can switch back also needs to early when
it runs out of substitutive content.

However, make the corresponding changes to the
packet loss fields in case that insertion duration is longer than report block besides the reserved
gap duration, there exists SSRC field and the
extended highest sequence number field.

When the mixer receives an overlap of RTCP extended report (XR) block, it should
rewrite the substitutive RTP stream
and XR report block in a similar way to the reception report
block in the RTCP report.

The mixer can also inform the main RTP stream at splicing out point. One straightforward
approach is that sender or the substitutive RTP
sender of the reception quality of the content reaches the mixer takes a ungracefule action, terminating
during the
splicing and switching back time when the content is not sent to main the RTP stream even if this may cause
media stuttering on receiver. There
This is an alternative approach which
may be mild but somewhat complex, done by the mixer buffers main content generating RTCP reports for a
while until the main RTP
stream and/or the substitutive RTP stream. These RTCP reports use
the SSRC of the mixer. If the substitutive content is finished, and then transmits
buffered main comes from local
media file storage, the mixer does not need to generate RTCP reports
for the substitutive stream.

Based on above RTCP operating mechanism, the RTP sender whose content
is being passed to receiver at an acceleated bitrate (as
compared to will see the nominal bitrate reception quality of main RTP stream) until its buffer
level returns to normal. At this point in time, mixer transmits main
stream as received by the mixer, and the reception quality of spliced
stream as received by the receiver. The RTP sender whose content is
not being passed to receiver at an nominal bitrate of main RTP stream. Note
that mixer should take into account a variety will only see the reception quality of parameters, such
its stream as
available bandwidth between mixer and receiver, received by the mixer.

The mixer buffer level must forward RTCP SDES and receiver buffer level, to count BYE packets from the accelerated bitrate value.

Another reason to cause media clipping is synchronization delay at
splicing point if RTP receiver needs to synchronize multiple current
streams for playback. How to address this issue is discussed
the sender, and may forward them in
detail inverse direction as defined in [RFC6051], which provides three feasible approaches to
reduce synchronization delay.

4.4. Congestion Control Considerations

Provided that the substitutive content has somewhat different
characteristics to
section 7.3 of [RFC3550].

Once the main content mixer receives an RTP/AVPF [RFC4585] transport layer
feedback packet, it replaces (e.g., must handle it carefully as the more
dynamic content, feedback packet
may contain the information of the higher bandwidth occupation), or substitutive content may be encoded with different codec and has that come from different
encoding bitrate, some challenge raise
RTP senders. In this case the mixer needs to network capacity and
receiver buffer size. A more dynamic content or a higher encoding
bitrate stream might overload the network and possibly exceed divide the
receiver's media consumption rate, which might flood receiver's
buffer and eventually result in a buffer overflow. Either network
overload or buffer overflow would induce network congestion and
congestion-caused feedback
packet loss.

To be robust to network congestion into two separate feedback packets and packet loss, mixer must
continuously monitor the network situation by means of a variety of
manners:

1. RTCP receiver reports indicate packet loss [RFC3550].

2. RTCP NACKs for lost packet recovery [RFC4585].

3. RTCP ECN Feedback information [I-D.ietf-avtcore-ecn-for-rtp].

Upon detection of above three types of RTCP reports during splicing,
mixer will treat them with three different manners as following:

1. If mixer receives the RTCP receiver reports with packet loss
indication, it will process them as the description given information
in
section 7.3 of [RFC3550].

2. If mixer receives the RTCP NACK packets defined feedback control information (FCI) in [RFC4585] from
RTP receiver for packet loss recovery, it first identifies the
content category of lost packets to which two feedback
packets, just as the NACK corresponds.
Then, mixer will generate new RTCP NACK for the lost packets with
its own SSRC, and make corresponding changes to their sequence
numbers to match original, pre-spliced, packets. report process described above.

If the lost substitutive content comes from local media file storage, storage
(i.e., the mixer
acting can be regarded as substitutive media source will directly fetch the lost substitutive content and retransmit it to RTP receiver.

It is somewhat complex that the lost packets requested in a
single sender), any
RTCP NACK message not only contain the main content but
also packets received from downstream relate to the substitutive content. To address this, mixer
content must
divide be terminated on the RTCP NACK packet into two separate RTCP NACK packets:
one mixer without any further
processing.

4.3. Media Clipping Considerations

This section provides informative guideline about how media clipping
is shaped and how the mixer deal with the media clipping.

If the time slot for substitutive content mismatches (is shorter or
longer than) the duration of the main content to be replaced, then
media clipping may occur at the splicing point.

If the substitutive content has shorter duration from the main
content, then there will be a gap in the output RTP stream. The RTP
sequence number will be contiguous across this gap, but there will be
an unexpected jump in the RTP timestamp. This gap will cause the
receiver to have nothing to play. This is unavoidable, unless the
mixer adjusts the splice in or splice out point to compensate,
sending more of the main RTP stream in place of the shorter
substitutive stream, or unless the mixer can vary the length of the
substitutive content. It is the responsibility of the higher layer
protocols to ensure that the substitutive content is of the same
duration as the main content to be replaced.

If the insertion duration is longer than the reserved gap duration,
there will be an overlap between the substitutive RTP stream and the
main RTP stream at splicing out point. One straightforward approach
is that the mixer takes an ungraceful action, terminating the
splicing and switching back to main RTP stream even if this may cause
media stuttering on receiver. Alternatively, the splicer may
transcode the substitutive content to play at a faster rate than
normal, to adjust it to the length of the gap in the main content,
and generate a new RTP stream for the transcoded content. This is a
complex operation, and very specific to the content and media codec
used.

4.4. Congestion Control Considerations

If the substitutive content has somewhat different characteristics
from the main content it replaces, or if the substitutive content is
encoded with a different codec or has different encoding bitrate, it
might overload the network and might cause network congestion on the
path between the mixer and the RTP receiver(s) that would not have
been caused by the main content.

To be robust to network congestion and packet loss, a mixer that is
performing splicing must continuously monitor the status of
downstream network by monitoring any of the following RTCP reports
that are used:

1. RTCP receiver reports indicate packet loss [RFC3550].

2. RTCP NACKs for lost packet recovery [RFC4585].

3. RTCP ECN Feedback information [I-D.ietf-avtcore-ecn-for-rtp].

Once the mixer detects congestion on its downstream link, it will
treat these reports as follows:

1. If the mixer receives the RTCP receiver reports with packet loss
indication, it will forward the reports to the substitutive RTP
sender or the main RTP sender as described in section 4.2.

2. If mixer receives the RTCP NACK packets defined in [RFC4585] from
RTP receiver for packet loss recovery, it first identifies the
content category of lost packets to which the NACK corresponds.
Then, the mixer will generate new RTCP NACK for the lost packets
with its own SSRC, and make corresponding changes to their
sequence numbers to match original, pre-spliced, packets. If the
lost substitutive content comes from local media file storage,
the mixer acting as substitutive RTP sender will directly fetch
the lost substitutive content and retransmit it to RTP receiver.
The mixer may buffer the sent RTP packets and do the
retransmission.

It is somewhat complex that the lost packets requested in a
single RTCP NACK message not only contain the main content but
also the substitutive content. To address this, the mixer must
divide the RTCP NACK packet into two separate RTCP NACK packets:
one requests for the lost main content, and another requests for
the lost substitutive content.

3. In [I-D.ietf-avtcore-ecn-for-rtp], two RTCP extensions are
defined for ECN feedback: RTP/AVPF transport layer ECN feedback
packet for urgent ECN information, and RTCP XR ECN summary report
block for regular reporting of the ECN marking information. If an ECN-aware mixer receives any RTCP ECN feedback (i.e., RTCP feedbacks (RTCP ECN
feedback packets or RTCP XR summary reports) defined in
[I-D.ietf-avtcore-ecn-for-rtp] from the RTP receiver, it must
process them in a similar way to the RTP/AVPF feedback packet or
RTCP XR process described in section 4.2 of this memo.

These three methods require the mixer to run a congestion control
loop and bitrate adaptation between itself and RTP receiver. The
mixer can thin or transcode the main RTP stream or the substitutive
RTP
receiver, it must operates as description given stream, but such operations are very inefficient and difficult,
and bring undesirable delay. Fortunately in section 8.4 of
[I-D.ietf-avtcore-ecn-for-rtp], terminating this memo, the mixer
acting as splicer can rewrite the RTCP ECN feedback packets sent from downstream receivers, the RTP
receiver and forward them to the RTP sender, letting the RTP sender
knows that congestion is being experienced on the path between the
mixer and driving the RTP receiver. Then, the RTP sender applies its
congestion control
loop algorithm and reduces the media bitrate to a value
that is in compliance with congestion control principles for the
slowest link. The congestion control algorithm may be a TCP-friendly
bitrate adaptation between itself and downstream
receiver algorithm specified in [RFC5348], or a DCCP
congestion control algorithms defined in [RFC5762].

If the substitutive content comes from local media file storage, the
mixer must directly reduce the bitrate as if it were the media source. In addition, an ECN-
aware substitutive
RTP mixer must generate RTCP ECN feedback relating sender.

From above analysis, to reduce the
input RTP streams it terminates, and driving risk of congestion control
loop and remain the
bandwidth consumption stable over time, the substitutive RTP stream
is recommended to be encoded at an appropriate bitrate adaptation between itself to match that
of main RTP stream. If the substitutive RTP stream comes from the
substitutive RTP sender, this sender had better has some knowledge
about the media encoding bitrate of main content in advance. How it
knows that is out of scope in this draft.

4.5. Processing Splicing in User Invisibility Case

If it is desirable to prevent receivers from detecting that splicing
has occurred at the RTP layer, the mixer must not include a CSRC list
in outgoing RTP packets, and upstream must not forward RTCP from the main RTP
sender as
if it were or from the substitutive RTP sender.

Once mixer learns that congestion is being experienced on its
downstream link by means of above three detection mechanisms, it
should adapt Due to the bitrate absence of output stream in response to network
congestion. The bitrate adaptation may be determined by a TCP-
friendly bitrate adaptation algorithm specified in [RFC5348], or by a
DCCP congestion control algorithms defined
CSRC list in [RFC5762].

In practice, during splicing, the real reason output RTP stream, the RTP receiver only initiates
SDES, BYE and APP packets to cause congestion
usually is the different characteristic mixer without any knowledge of substitutive RTP stream
(more dynamic content or higher encoding bitrate) with the
main RTP
stream, sender and that stream transcoding or thinning on mixer the substitutive RTP sender.

CSRC list identifies the contributing sources, these SSRC identifiers
of contributing sources are kept globally unique for each RTP
session. The uniqueness of SSRC identifier is very
inefficient used to resolve
collisions and difficult operation. Therefore, detecting RTP-level forwarding loops as defined in
section 8.2 of [RFC3550]. The absence of CSRC list in this case will
create a means danger that enables
substitutive media source loops involving those contributing sources could
not be detected. So Non-RTP means must be used to limit detect and resolve
loops if the media bitrate it is currently
generating even in splicer does not add a CSRC list.

5. Implementation Considerations

When the absence of congestion mixer is used to handle RTP splicing, RTP receiver does not
need any RTP/RTCP extension for splicing. As a trade-off, additional
overhead could be induced on the path between
itself mixer which uses its own sequence
number space and timing model. So the mixer will rewrite RTP
sequence number and timestamp whatever splicing is desirable. The TMMBR message defined in
[RFC5104] provides an effective method. When active or not, and
generate RTCP flows for both sides. In case the mixer detects
congestion on its downstream link during splicing, it uses TMMBR to
request substitutive media source serves
multiple main RTP streams simultaneously, this may lead to reduce more
overhead on the media bitrate to a
value that mixer.

If User Invisibility Requirement is required, CSRC list is not
included in compliance outgoing RTP packet, this brings a potential issue with congestion control principles for
loop detection as briefly described in section 4.5.

6. Security Considerations

The splicing application is subject to the slowest link. Upon reception general security
considerations of TMMBR, substitutive media source
applies its congestion control algorithm and responds Temporary
Maximum Media Stream Bit Rate Notification (TMMBN) to mixer.

If the substitutive content comes from local media file storage, RTP specification [RFC3550].

The mixer must directly reduce the substitutive media bitrate acting as splicer replace some content with other content
in RTP packets, thus breaking the
substitutive media end-to-end security, such as
integrity protection and source when it detects any congestion on its
downstream link during splicing.

From above analysis, authentication. Its behavior looks
like a middleman attack, but SRTP [RFC3711] can be used to reduce
authenticate the risk of congestion mixer, and remain provide integrity protection on the
bandwidth consumption stable over time, path
between the substitutive RTP stream mixer and the receivers, but the receiver cannot (and is recommended
not supposed to be encoded at an appropriate bitrate to match that
of main RTP stream. If able to) determine what content comes from the substitutive
main RTP stream sender and what comes from the substitutive media source, RTP sender by
looking at the source had better has some knowledge
about RTP layer.

The RTP receiver does not communicate directly with the media encoding bitrate of main content in advance. How it
knows that is out of scope in this draft.

4.5. Processing Splicing in User Invisibility Case

Mixer will not includes CRSC list in outgoing RTP packets to prevent
user from detecting
sender or the splicing occurred on substitutive RTP level. Due to sender, and does not have an end-to-
end security relationship with them at the
absence RTP layer. The nature of CRSC list in current
this RTP stream, service offered by a network operator employing a content
splicer is that the RTP layer security relationship is between the
receiver only
initiates SDES, BYE and APP packets to mixer without any knowledge of
main media source the mixer, and between the senders and the mixer, and substitutive media source. This creates a
danger that loops involving those sources could
not be detected.

5. Implementation Considerations

When mixer is used end-to-end. The network operator must delegate authority to handle RTP splicing, RTP receiver does not need
any RTP/RTCP extension for splicing. As a trade-off, additional
overhead could be induced on mixer which uses its own sequence number
space and timing model. So the
mixer will rewrite in exchange for the ability to perform RTP sequence number
and timestamp whatever splicing inside the
network.

If encryption is active or not, and generate RTCP
flows for both sides. In case employed, the mixer serves multiple main RTP streams
simultaneously, this may lead to more overhead on mixer.

In addition, there is a potential issue with loop detection, which
would must be problematic if User Invisibility Requirement is required.

6. Security Considerations able to decrypt the
inbound RTP packets and re-encrypt the outbound RTP packets.

If any payload internal security mechanisms (e.g., ISMACryp
[ISMACryp]) are used, only media source the RTP sender and the RTP receiver can
learn the security keying material generated by such internal
security mechanism, in which case, any middlebox (e.g., mixer)
between media source the RTP sender and the RTP receiver can't get such keying material. Only when regular transport
security mechanisms (e.g., SRTP, IPSec, etc) are used, mixer will
process the packets passing through it.

The security considerations of the RTP specification [RFC3550], the
Extended RTP profile for RTCP-Based Feedback [RFC4585], and the
Secure Real-time Transport Protocol [RFC3711] apply. Mixer must be
trusted by main media source and insertion media source,
material, and must be
included in the security context. thus fail to perform splicing.

7. IANA Considerations

No IANA actions are required.

8. Acknowledgments

The following individuals have reviewed the earlier versions of this
specification and provided very valuable comments: Colin Perkins,
Magnus Westerlund, Roni Even, Tom Van Caenegem, Joerg Ott, David R
Oran, Cullen Jennings, Ali C Begen, Charles Eckel and Ning Zong.

9. Change Log

9.1. draft-xia-avtext-splicing-for-rtp-01

The following are the major changes compared to previous version 00:

o Use 10. Appendix- Why Mixer Is Chosen

Translator and mixer to handle both user visible and invisible splicing.

o Add one subsection to describe media clipping considerations.

o Add one subsection to describe congestion control considerations.

9.2. draft-xia-avtext-splicing-for-rtp-00

The following are the major changes compared to previous AVT I-D
version 00:

o Change primary can realize splicing by changing a set of
RTP stream parameters.

Translator has no SSRC, hence it is transparent to main RTP stream, add current sender and
receiver. Therefore, RTP
stream as sender sees the streaming received by RTP receiver.

o Eliminate full path to the ambiguity of inserted content with substitutive
content which replaces receiver
when translator is passing its content. When translator insert the main
substitutive content rather than pause it.

o Clarify RTP sender could get a report on the signaling requirements.

o Delete path up to
translator itself. Additionally, if user detectability is not
required, translator does not need to rewrite RTP headers, the description on Mixer and MCU in section 4, mainly focus
overhead on the direction whether a Translator translator can act as a Splicer.

o Add section 5 be avoided.

If mixer is used to describe do splicing, it can also allow RTP sender to
learn the exact guidance situation of its content on how an RTP
Translator receiver or on mixer just like
translator does, which is used specified in section 4.2. Compared to
translator, mixer's outstanding benefit is that it is pretty straight
forward to do with bit-rate adaptation to handle splicing.

o Modify the security varying network
conditions. But translator needs more considerations section and add acknowledges
section. its
implementation is more complex.

From above analysis, both translator and mixer have their own
advantages: less overhead or less complexity on handling RTCP.
Through long and sophisticated discussion, the avtext WG members
prefer less complexity rather than less overhead and incline to mixer
to do splicing.

If one chooses mixer as splicer, the overhead on mixer must be taken
into account. If one chooses translator as splicer, the complex RTCP
processing on translator must be taken into account.

10. References

10.1. Normative References

[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.

[RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
Norrman, "The Secure Real-time Transport Protocol (SRTP)",
RFC 3711, March 2004.

[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
July 2006.

[RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
"Codec Control Messages in the RTP Audio-Visual Profile
with Feedback (AVPF)", RFC 5104, February 2008.

[RFC6051] Perkins, C. and T. Schierl, "Rapid Synchronisation of RTP
Flows", RFC 6051, November 2010.

[I-D.ietf-avtcore-ecn-for-rtp]
Westerlund, M., "Explicit Congestion Notification (ECN)
for RTP over UDP", draft-ietf-avtcore-ecn-for-rtp-06 draft-ietf-avtcore-ecn-for-rtp-08 (work
in progress), February May 2012.

10.2. Informative References

[RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP
Friendly Rate Control (TFRC): Protocol Specification",
RFC 5348, September 2008.

[RFC5762] Perkins, C., "RTP and the Datagram Congestion Control
Protocol (DCCP)", RFC 5762, April 2010.

[SCTE30] Society of Cable Telecommunications Engineers (SCTE),
"Digital Program Insertion Splicing API", 2001. 2009.

[SCTE35] Society of Cable Telecommunications Engineers (SCTE),
"Digital Program Insertion Cueing Message for Cable",
2004.
2011.

[ISMACryp]
Internet Streaming Media Alliance (ISMA), "ISMA Encryption
and Authentication Specification 2.0", November 2007.

Author's Address

Jinwei Xia
Huawei
Software No.101
Nanjing, Yuhuatai District 210012
China

Phone: +86-025-86622310
Email: xiajinwei@huawei.com