draft-ietf-avtext-rtp-grouping-taxonomy-08.txt   rfc7656.txt 
Network Working Group J. Lennox Internet Engineering Task Force (IETF) J. Lennox
Internet-Draft Vidyo Request for Comments: 7656 Vidyo
Intended status: Informational K. Gross Category: Informational K. Gross
Expires: January 21, 2016 AVA ISSN: 2070-1721 AVA
S. Nandakumar S. Nandakumar
G. Salgueiro G. Salgueiro
Cisco Systems Cisco Systems
B. Burman, Ed. B. Burman, Ed.
Ericsson Ericsson
July 20, 2015 November 2015
A Taxonomy of Semantics and Mechanisms for Real-Time Transport Protocol A Taxonomy of Semantics and Mechanisms for
(RTP) Sources Real-Time Transport Protocol (RTP) Sources
draft-ietf-avtext-rtp-grouping-taxonomy-08
Abstract Abstract
The terminology about, and associations among, Real-Time Transport The terminology about, and associations among, Real-time Transport
Protocol (RTP) sources can be complex and somewhat opaque. This Protocol (RTP) sources can be complex and somewhat opaque. This
document describes a number of existing and proposed properties and document describes a number of existing and proposed properties and
relationships among RTP sources, and defines common terminology for relationships among RTP sources and defines common terminology for
discussing protocol entities and their relationships. discussing protocol entities and their relationships.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This document is not an Internet Standards Track specification; it is
provisions of BCP 78 and BCP 79. published for informational purposes.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months This document is a product of the Internet Engineering Task Force
and may be updated, replaced, or obsoleted by other documents at any (IETF). It represents the consensus of the IETF community. It has
time. It is inappropriate to use Internet-Drafts as reference received public review and has been approved for publication by the
material or to cite them other than as "work in progress." Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are a candidate for any level of Internet
Standard; see Section 2 of RFC 5741.
This Internet-Draft will expire on January 21, 2016. Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc7656.
Copyright Notice Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the Copyright (c) 2015 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1. Media Chain . . . . . . . . . . . . . . . . . . . . . . . 5 2.1. Media Chain . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1. Physical Stimulus . . . . . . . . . . . . . . . . . . 9 2.1.1. Physical Stimulus . . . . . . . . . . . . . . . . . . 10
2.1.2. Media Capture . . . . . . . . . . . . . . . . . . . . 9 2.1.2. Media Capture . . . . . . . . . . . . . . . . . . . . 10
2.1.3. Raw Stream . . . . . . . . . . . . . . . . . . . . . 9 2.1.3. Raw Stream . . . . . . . . . . . . . . . . . . . . . 10
2.1.4. Media Source . . . . . . . . . . . . . . . . . . . . 10 2.1.4. Media Source . . . . . . . . . . . . . . . . . . . . 11
2.1.5. Source Stream . . . . . . . . . . . . . . . . . . . . 10 2.1.5. Source Stream . . . . . . . . . . . . . . . . . . . . 11
2.1.6. Media Encoder . . . . . . . . . . . . . . . . . . . . 11 2.1.6. Media Encoder . . . . . . . . . . . . . . . . . . . . 12
2.1.7. Encoded Stream . . . . . . . . . . . . . . . . . . . 12 2.1.7. Encoded Stream . . . . . . . . . . . . . . . . . . . 13
2.1.8. Dependent Stream . . . . . . . . . . . . . . . . . . 12 2.1.8. Dependent Stream . . . . . . . . . . . . . . . . . . 13
2.1.9. Media Packetizer . . . . . . . . . . . . . . . . . . 12 2.1.9. Media Packetizer . . . . . . . . . . . . . . . . . . 13
2.1.10. RTP Stream . . . . . . . . . . . . . . . . . . . . . 13 2.1.10. RTP Stream . . . . . . . . . . . . . . . . . . . . . 14
2.1.11. RTP-based Redundancy . . . . . . . . . . . . . . . . 13 2.1.11. RTP-Based Redundancy . . . . . . . . . . . . . . . . 14
2.1.12. Redundancy RTP Stream . . . . . . . . . . . . . . . . 14 2.1.12. Redundancy RTP Stream . . . . . . . . . . . . . . . . 15
2.1.13. RTP-based Security . . . . . . . . . . . . . . . . . 14 2.1.13. RTP-Based Security . . . . . . . . . . . . . . . . . 15
2.1.14. Secured RTP Stream . . . . . . . . . . . . . . . . . 15 2.1.14. Secured RTP Stream . . . . . . . . . . . . . . . . . 16
2.1.15. Media Transport . . . . . . . . . . . . . . . . . . . 15 2.1.15. Media Transport . . . . . . . . . . . . . . . . . . . 16
2.1.16. Media Transport Sender . . . . . . . . . . . . . . . 16 2.1.16. Media Transport Sender . . . . . . . . . . . . . . . 17
2.1.17. Sent RTP Stream . . . . . . . . . . . . . . . . . . . 17 2.1.17. Sent RTP Stream . . . . . . . . . . . . . . . . . . . 18
2.1.18. Network Transport . . . . . . . . . . . . . . . . . . 17 2.1.18. Network Transport . . . . . . . . . . . . . . . . . . 18
2.1.19. Transported RTP Stream . . . . . . . . . . . . . . . 17 2.1.19. Transported RTP Stream . . . . . . . . . . . . . . . 18
2.1.20. Media Transport Receiver . . . . . . . . . . . . . . 17 2.1.20. Media Transport Receiver . . . . . . . . . . . . . . 18
2.1.21. Received Secured RTP Stream . . . . . . . . . . . . . 18 2.1.21. Received Secured RTP Stream . . . . . . . . . . . . . 19
2.1.22. RTP-based Validation . . . . . . . . . . . . . . . . 18 2.1.22. RTP-Based Validation . . . . . . . . . . . . . . . . 19
2.1.23. Received RTP Stream . . . . . . . . . . . . . . . . . 18 2.1.23. Received RTP Stream . . . . . . . . . . . . . . . . . 19
2.1.24. Received Redundancy RTP Stream . . . . . . . . . . . 18 2.1.24. Received Redundancy RTP Stream . . . . . . . . . . . 19
2.1.25. RTP-based Repair . . . . . . . . . . . . . . . . . . 18 2.1.25. RTP-Based Repair . . . . . . . . . . . . . . . . . . 19
2.1.26. Repaired RTP Stream . . . . . . . . . . . . . . . . . 18 2.1.26. Repaired RTP Stream . . . . . . . . . . . . . . . . . 19
2.1.27. Media Depacketizer . . . . . . . . . . . . . . . . . 19 2.1.27. Media Depacketizer . . . . . . . . . . . . . . . . . 20
2.1.28. Received Encoded Stream . . . . . . . . . . . . . . . 19 2.1.28. Received Encoded Stream . . . . . . . . . . . . . . . 20
2.1.29. Media Decoder . . . . . . . . . . . . . . . . . . . . 19 2.1.29. Media Decoder . . . . . . . . . . . . . . . . . . . . 20
2.1.30. Received Source Stream . . . . . . . . . . . . . . . 19 2.1.30. Received Source Stream . . . . . . . . . . . . . . . 20
2.1.31. Media Sink . . . . . . . . . . . . . . . . . . . . . 19 2.1.31. Media Sink . . . . . . . . . . . . . . . . . . . . . 21
2.1.32. Received Raw Stream . . . . . . . . . . . . . . . . . 20 2.1.32. Received Raw Stream . . . . . . . . . . . . . . . . . 21
2.1.33. Media Render . . . . . . . . . . . . . . . . . . . . 20 2.1.33. Media Render . . . . . . . . . . . . . . . . . . . . 21
2.2. Communication Entities . . . . . . . . . . . . . . . . . 20 2.2. Communication Entities . . . . . . . . . . . . . . . . . 22
2.2.1. Endpoint . . . . . . . . . . . . . . . . . . . . . . 22 2.2.1. Endpoint . . . . . . . . . . . . . . . . . . . . . . 23
2.2.2. RTP Session . . . . . . . . . . . . . . . . . . . . . 22 2.2.2. RTP Session . . . . . . . . . . . . . . . . . . . . . 23
2.2.3. Participant . . . . . . . . . . . . . . . . . . . . . 23 2.2.3. Participant . . . . . . . . . . . . . . . . . . . . . 24
2.2.4. Multimedia Session . . . . . . . . . . . . . . . . . 23 2.2.4. Multimedia Session . . . . . . . . . . . . . . . . . 24
2.2.5. Communication Session . . . . . . . . . . . . . . . . 24 2.2.5. Communication Session . . . . . . . . . . . . . . . . 25
3. Concepts of Inter-Relations . . . . . . . . . . . . . . . . . 24 3. Concepts of Inter-Relations . . . . . . . . . . . . . . . . . 25
3.1. Synchronization Context . . . . . . . . . . . . . . . . . 24 3.1. Synchronization Context . . . . . . . . . . . . . . . . . 26
3.1.1. RTCP CNAME . . . . . . . . . . . . . . . . . . . . . 25 3.1.1. RTCP CNAME . . . . . . . . . . . . . . . . . . . . . 26
3.1.2. Clock Source Signaling . . . . . . . . . . . . . . . 25 3.1.2. Clock Source Signaling . . . . . . . . . . . . . . . 26
3.1.3. Implicitly via RtcMediaStream . . . . . . . . . . . . 25 3.1.3. Implicitly via RtcMediaStream . . . . . . . . . . . . 26
3.1.4. Explicitly via SDP Mechanisms . . . . . . . . . . . . 25 3.1.4. Explicitly via SDP Mechanisms . . . . . . . . . . . . 26
3.2. Endpoint . . . . . . . . . . . . . . . . . . . . . . . . 25 3.2. Endpoint . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3. Participant . . . . . . . . . . . . . . . . . . . . . . . 26 3.3. Participant . . . . . . . . . . . . . . . . . . . . . . . 27
3.4. RtcMediaStream . . . . . . . . . . . . . . . . . . . . . 26 3.4. RtcMediaStream . . . . . . . . . . . . . . . . . . . . . 27
3.5. Multi-Channel Audio . . . . . . . . . . . . . . . . . . . 26 3.5. Multi-Channel Audio . . . . . . . . . . . . . . . . . . . 28
3.6. Simulcast . . . . . . . . . . . . . . . . . . . . . . . . 27 3.6. Simulcast . . . . . . . . . . . . . . . . . . . . . . . . 28
3.7. Layered Multi-Stream . . . . . . . . . . . . . . . . . . 28 3.7. Layered Multi-Stream . . . . . . . . . . . . . . . . . . 30
3.8. RTP Stream Duplication . . . . . . . . . . . . . . . . . 29 3.8. RTP Stream Duplication . . . . . . . . . . . . . . . . . 32
3.9. Redundancy Format . . . . . . . . . . . . . . . . . . . . 30 3.9. Redundancy Format . . . . . . . . . . . . . . . . . . . . 33
3.10. RTP Retransmission . . . . . . . . . . . . . . . . . . . 31 3.10. RTP Retransmission . . . . . . . . . . . . . . . . . . . 33
3.11. Forward Error Correction . . . . . . . . . . . . . . . . 33 3.11. Forward Error Correction . . . . . . . . . . . . . . . . 35
3.12. RTP Stream Separation . . . . . . . . . . . . . . . . . . 34 3.12. RTP Stream Separation . . . . . . . . . . . . . . . . . . 36
3.13. Multiple RTP Sessions over one Media Transport . . . . . 35 3.13. Multiple RTP Sessions over one Media Transport . . . . . 37
4. Mapping from Existing Terms . . . . . . . . . . . . . . . . . 35 4. Mapping from Existing Terms . . . . . . . . . . . . . . . . . 37
4.1. Telepresence Terms . . . . . . . . . . . . . . . . . . . 35 4.1. Telepresence Terms . . . . . . . . . . . . . . . . . . . 37
4.1.1. Audio Capture . . . . . . . . . . . . . . . . . . . . 35 4.1.1. Audio Capture . . . . . . . . . . . . . . . . . . . . 37
4.1.2. Capture Device . . . . . . . . . . . . . . . . . . . 35 4.1.2. Capture Device . . . . . . . . . . . . . . . . . . . 37
4.1.3. Capture Encoding . . . . . . . . . . . . . . . . . . 36 4.1.3. Capture Encoding . . . . . . . . . . . . . . . . . . 38
4.1.4. Capture Scene . . . . . . . . . . . . . . . . . . . . 36 4.1.4. Capture Scene . . . . . . . . . . . . . . . . . . . . 38
4.1.5. Endpoint . . . . . . . . . . . . . . . . . . . . . . 36 4.1.5. Endpoint . . . . . . . . . . . . . . . . . . . . . . 38
4.1.6. Individual Encoding . . . . . . . . . . . . . . . . . 36 4.1.6. Individual Encoding . . . . . . . . . . . . . . . . . 38
4.1.7. Media Capture . . . . . . . . . . . . . . . . . . . . 36 4.1.7. Media Capture . . . . . . . . . . . . . . . . . . . . 38
4.1.8. Media Consumer . . . . . . . . . . . . . . . . . . . 36 4.1.8. Media Consumer . . . . . . . . . . . . . . . . . . . 38
4.1.9. Media Provider . . . . . . . . . . . . . . . . . . . 37 4.1.9. Media Provider . . . . . . . . . . . . . . . . . . . 39
4.1.10. Stream . . . . . . . . . . . . . . . . . . . . . . . 37 4.1.10. Stream . . . . . . . . . . . . . . . . . . . . . . . 39
4.1.11. Video Capture . . . . . . . . . . . . . . . . . . . . 37 4.1.11. Video Capture . . . . . . . . . . . . . . . . . . . . 39
4.2. Media Description . . . . . . . . . . . . . . . . . . . . 37 4.2. Media Description . . . . . . . . . . . . . . . . . . . . 39
4.3. Media Stream . . . . . . . . . . . . . . . . . . . . . . 37 4.3. Media Stream . . . . . . . . . . . . . . . . . . . . . . 39
4.4. Multimedia Conference . . . . . . . . . . . . . . . . . . 37 4.4. Multimedia Conference . . . . . . . . . . . . . . . . . . 39
4.5. Multimedia Session . . . . . . . . . . . . . . . . . . . 38 4.5. Multimedia Session . . . . . . . . . . . . . . . . . . . 40
4.6. Multipoint Control Unit (MCU) . . . . . . . . . . . . . . 38 4.6. Multipoint Control Unit (MCU) . . . . . . . . . . . . . . 40
4.7. Multi-Session Transmission (MST) . . . . . . . . . . . . 38 4.7. Multi-Session Transmission (MST) . . . . . . . . . . . . 40
4.8. Recording Device . . . . . . . . . . . . . . . . . . . . 39 4.8. Recording Device . . . . . . . . . . . . . . . . . . . . 41
4.9. RtcMediaStream . . . . . . . . . . . . . . . . . . . . . 39 4.9. RtcMediaStream . . . . . . . . . . . . . . . . . . . . . 41
4.10. RtcMediaStreamTrack . . . . . . . . . . . . . . . . . . . 39 4.10. RtcMediaStreamTrack . . . . . . . . . . . . . . . . . . . 41
4.11. RTP Sender . . . . . . . . . . . . . . . . . . . . . . . 39 4.11. RTP Receiver . . . . . . . . . . . . . . . . . . . . . . 41
4.12. RTP Session . . . . . . . . . . . . . . . . . . . . . . . 39 4.12. RTP Sender . . . . . . . . . . . . . . . . . . . . . . . 41
4.13. Single Session Transmission (SST) . . . . . . . . . . . . 39 4.13. RTP Session . . . . . . . . . . . . . . . . . . . . . . . 41
4.14. SSRC . . . . . . . . . . . . . . . . . . . . . . . . . . 39 4.14. Single-Session Transmission (SST) . . . . . . . . . . . . 41
4.15. SSRC . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5. Security Considerations . . . . . . . . . . . . . . . . . . . 40 5. Security Considerations . . . . . . . . . . . . . . . . . . . 42
6. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . 40 6. Informative References . . . . . . . . . . . . . . . . . . . 42
7. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 40 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 45
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 41 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . 45
9. Informative References . . . . . . . . . . . . . . . . . . . 41 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 46
Appendix A. Changes From Earlier Versions . . . . . . . . . . . 44
A.1. Modifications Between WG Version -07 and -08 . . . . . . 44
A.2. Modifications Between WG Version -06 and -07 . . . . . . 45
A.3. Modifications Between WG Version -05 and -06 . . . . . . 45
A.4. Modifications Between WG Version -04 and -05 . . . . . . 46
A.5. Modifications Between WG Version -03 and -04 . . . . . . 46
A.6. Modifications Between WG Version -02 and -03 . . . . . . 47
A.7. Modifications Between WG Version -01 and -02 . . . . . . 47
A.8. Modifications Between WG Version -00 and -01 . . . . . . 48
A.9. Modifications Between Version -02 and -03 . . . . . . . . 48
A.10. Modifications Between Version -01 and -02 . . . . . . . . 48
A.11. Modifications Between Version -00 and -01 . . . . . . . . 48
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 49
1. Introduction 1. Introduction
The existing taxonomy of sources in the Real-Time Transport Protocol The existing taxonomy of sources in the Real-time Transport Protocol
(RTP) [RFC3550] has previously been regarded as confusing and (RTP) [RFC3550] has previously been regarded as confusing and
inconsistent. Consequently, a deep understanding of how the inconsistent. Consequently, a deep understanding of how the
different terms relate to each other becomes a real challenge. different terms relate to each other becomes a real challenge.
Frequently cited examples of this confusion are (1) how different Frequently cited examples of this confusion are (1) how different
protocols that make use of RTP use the same terms to signify protocols that make use of RTP use the same terms to signify
different things and (2) how the complexities addressed at one layer different things and (2) how the complexities addressed at one layer
are often glossed over or ignored at another. are often glossed over or ignored at another.
This document improves clarity by reviewing the semantics of various This document improves clarity by reviewing the semantics of various
aspects of sources in RTP. As an organizing mechanism, it approaches aspects of sources in RTP. As an organizing mechanism, it approaches
this by describing various ways that RTP sources are transformed on this by describing various ways that RTP sources are transformed on
their way between sender and receiver, and how they can be grouped their way between sender and receiver, and how they can be grouped
and associated together. and associated together.
All non-specific references to ControLling mUltiple streams for All non-specific references to ControLling mUltiple streams for
tElepresence (CLUE) in this document map to [I-D.ietf-clue-framework] tElepresence (CLUE) in this document map to [CLUE-FRAME], and all
and all references to Web Real-Time Communications (WebRTC) map to references to Web Real-time Communications (WebRTC) map to
[I-D.ietf-rtcweb-overview]. [WEBRTC-OVERVIEW].
2. Concepts 2. Concepts
This section defines concepts that serve to identify and name various This section defines concepts that serve to identify and name various
transformations and streams in a given RTP usage. For each concept, transformations and streams in a given RTP usage. For each concept,
alternate definitions and usages that co-exist today are listed along alternate definitions and usages that coexist today are listed along
with various characteristics that further describes the concept. with various characteristics that further describe the concept.
These concepts are divided into two categories, one related to the These concepts are divided into two categories: one is related to the
chain of streams and transformations that media can be subject to, chain of streams and transformations that Media can be subject to,
the other for entities involved in the communication. and the other is for entities involved in the communication.
2.1. Media Chain 2.1. Media Chain
In the context of this document, Media is a sequence of synthetic or In the context of this document, media is a sequence of synthetic or
Physical Stimuli (Section 2.1.1) (sound waves, photons, key-strokes), Physical Stimuli (Section 2.1.1) -- for example, sound waves,
represented in digital form. Synthesized Media is typically photons, key strokes -- represented in digital form. Synthesized
generated directly in the digital domain. media is typically generated directly in the digital domain.
This section contains the concepts that can be involved in taking This section contains the concepts that can be involved in taking
Media at a sender side and transporting it to a receiver, which may media at a sender side and transporting it to a receiver, which may
recover a sequence of physical stimuli. This chain of concepts is of recover a sequence of physical stimuli. This chain of concepts is of
two main types, streams and transformations. Streams are time-based two main types: streams and transformations. Streams are time-based
sequences of samples of the physical stimulus in various sequences of samples of the physical stimulus in various
representations, while transformations changes the representation of representations, while transformations change the representation of
the streams in some way. the streams in some way.
The below examples are basic ones and it is important to keep in mind The below examples are basic ones, and it is important to keep in
that this conceptual model enables more complex usages. Some will be mind that this conceptual model enables more complex usages. Some
further discussed in later sections of this document. In general the will be further discussed in later sections of this document. In
following applies to this model: general the following applies to this model:
o A transformation may have zero or more inputs and one or more o A transformation may have zero or more inputs and one or more
outputs. outputs.
o A stream is of some type, such as audio, video, real-time text, o A stream is of some type, such as audio, video, real-time text,
etc. etc.
o A stream has one source transformation and one or more sink o A stream has one source transformation and one or more sink
transformations (with the exception of Physical Stimulus transformations (with the exception of physical stimulus
(Section 2.1.1) that may lack source or sink transformation). (Section 2.1.1) that may lack source or sink transformation).
o Streams can be forwarded from a transformation output to any o Streams can be forwarded from a transformation output to any
number of inputs on other transformations that support that type. number of inputs on other transformations that support that type.
o If the output of a transformation is sent to multiple o If the output of a transformation is sent to multiple
transformations, those streams will be identical; it takes a transformations, those streams will be identical; it takes a
transformation to make them different. transformation to make them different.
o There are no formal limitations on how streams are connected to o There are no formal limitations on how streams are connected to
transformations. transformations.
It is also important to remember that this is a conceptual model. It is also important to remember that this is a conceptual model.
Thus real-world implementations may look different and have different Thus, real-world implementations may look different and have a
structure. different structure.
To provide a basic understanding of the relationships in the chain we To provide a basic understanding of the relationships in the chain,
first introduce the concepts for the sender side (Figure 1). This we first introduce the concepts for the sender side (Figure 1). This
covers physical stimuli until media packets are emitted onto the covers physical stimuli until media packets are emitted onto the
network. network.
Physical Stimulus Physical Stimulus
| |
V V
+----------------------+ +----------------------+
| Media Capture | | Media Capture |
+----------------------+ +----------------------+
| |
skipping to change at page 6, line 32 skipping to change at page 7, line 27
| |
Source Stream Source Stream
V V
+----------------------+ +----------------------+
| Media Encoder | | Media Encoder |
+----------------------+ +----------------------+
| |
Encoded Stream +------------+ Encoded Stream +------------+
V | V V | V
+----------------------+ | +----------------------+ +----------------------+ | +----------------------+
| Media Packetizer | | | RTP-based Redundancy | | Media Packetizer | | | RTP-Based Redundancy |
+----------------------+ | +----------------------+ +----------------------+ | +----------------------+
| | | | | |
+-------------+ Redundancy RTP Stream +-------------+ Redundancy RTP Stream
Source RTP Stream | Source RTP Stream |
V V V V
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| RTP-based Security | | RTP-based Security | | RTP-Based Security | | RTP-Based Security |
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| | | |
Secured RTP Stream Secured Redundancy RTP Stream Secured RTP Stream Secured Redundancy RTP Stream
V V V V
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| Media Transport | | Media Transport | | Media Transport | | Media Transport |
+----------------------+ +----------------------+ +----------------------+ +----------------------+
Figure 1: Sender Side Concepts in the Media Chain Figure 1: Sender Side Concepts in the Media Chain
In Figure 1 we have included a branched chain to cover the concepts In Figure 1, we have included a branched chain to cover the concepts
for using redundancy to improve the reliability of the transport. for using redundancy to improve the reliability of the transport.
The Media Transport concept is an aggregate that is decomposed in The Media Transport concept is an aggregate that is decomposed in
Section 2.1.15. Section 2.1.15.
In Figure 2 we review a receiver media chain matching the sender In Figure 2, we review a receiver media chain matching the sender
side, to look at the inverse transformations and their attempts to side, to look at the inverse transformations and their attempts to
recover identical streams as in the sender chain, subject to what may recover identical streams as in the sender chain, subject to what may
be lossy compression and imperfect Media Transport. Note that the be lossy compression and imperfect media transport. Note that the
streams out of a reverse transformation, like the Source Stream out streams out of a reverse transformation, like the Source Stream out
the Media Decoder are in many cases not the same as the corresponding of the Media Decoder, are in many cases not the same as the
ones on the sender side, thus they are prefixed with a "Received" to corresponding ones on the sender side; thus, they are prefixed with a
denote a potentially modified version. The reason for not being the "received" to denote a potentially modified version. The reason for
same lies in the transformations that can be of irreversible type. not being the same lies in the transformations that can be of
For example, lossy source coding in the Media Encoder prevents the irreversible type. For example, lossy source coding in the Media
Source Stream out of the Media Decoder to be the same as the one fed Encoder prevents the source stream out of the media decoder from
into the Media Encoder. Other reasons include packet loss or late being the same as the one fed into the media encoder. Other reasons
loss in the Media Transport transformation that even RTP-based include packet loss in the media transport transformation that even
Repair, if used, fails to repair. However, some transformations are RTP-based Repair, if used, fails to repair.
not always present, like RTP-based Repair that cannot operate without
Redundancy RTP Streams.
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| Media Transport | | Media Transport | | Media Transport | | Media Transport |
+----------------------+ +----------------------+ +----------------------+ +----------------------+
Received | Received | Secured Received | Received | Secured
Secured RTP Stream Redundancy RTP Stream Secured RTP Stream Redundancy RTP Stream
V V V V
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| RTP-based Validation | | RTP-based Validation | | RTP-Based Validation | | RTP-Based Validation |
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| | | |
Received RTP Stream Received Redundancy RTP Stream Received RTP Stream Received Redundancy RTP Stream
| | | |
| +--------------------+ | +--------------------+
V V V V
+----------------------+ +----------------------+
| RTP-based Repair | | RTP-Based Repair |
+----------------------+ +----------------------+
| |
Repaired RTP Stream Repaired RTP Stream
V V
+----------------------+ +----------------------+
| Media Depacketizer | | Media Depacketizer |
+----------------------+ +----------------------+
| |
Received Encoded Stream Received Encoded Stream
V V
skipping to change at page 8, line 44 skipping to change at page 9, line 44
| |
Received Source Stream Received Source Stream
V V
+----------------------+ +----------------------+
| Media Sink |--> Synchronization Information | Media Sink |--> Synchronization Information
+----------------------+ +----------------------+
| |
Received Raw Stream Received Raw Stream
V V
+----------------------+ +----------------------+
| Media Renderer | | Media Render |
+----------------------+ +----------------------+
| |
V V
Physical Stimulus Physical Stimulus
Figure 2: Receiver Side Concepts of the Media Chain Figure 2: Receiver Side Concepts of the Media Chain
2.1.1. Physical Stimulus 2.1.1. Physical Stimulus
The Physical Stimulus is a physical event in the analog domain that The physical stimulus is a physical event in the analog domain that
can be sampled and converted to digital form by an appropriate sensor can be sampled and converted to digital form by an appropriate sensor
or transducer. This include sound waves making up audio, photons in or transducer. This includes sound waves making up audio, photons in
a light field, or other excitations or interactions with sensors, a light field, or other excitations or interactions with sensors,
like keystrokes on a keyboard. like keystrokes on a keyboard.
2.1.2. Media Capture 2.1.2. Media Capture
Media Capture is the process of transforming the analog Physical Media Capture is the process of transforming the analog physical
Stimulus (Section 2.1.1) into digital Media using an appropriate stimulus (Section 2.1.1) into digital media using an appropriate
sensor or transducer. The Media Capture performs a digital sampling sensor or transducer. The media capture performs a digital sampling
of the physical stimulus, usually periodically, and outputs this in of the physical stimulus, usually periodically, and outputs this in
some representation as a Raw Stream (Section 2.1.3). This data is some representation as a Raw Stream (Section 2.1.3). This data is
considered "Media", because it includes data that is periodically considered "media", because it includes data that is periodically
sampled, or made up of a set of timed asynchronous events. The Media sampled or made up of a set of timed asynchronous events. The media
Capture is normally instantiated in some type of device, i.e. media capture is normally instantiated in some type of device, i.e., media
capture device. Examples of different types of media capturing capture device. Examples of different types of media capturing
devices are digital cameras, microphones connected to A/D converters, devices are digital cameras, microphones connected to A/D converters,
or keyboards. or keyboards.
Characteristics: Characteristics:
o A Media Capture is identified either by hardware/manufacturer ID o A media capture is identified either by hardware/manufacturer ID
or via a session-scoped device identifier as mandated by the or via a session-scoped device identifier as mandated by the
application usage. application usage.
o A Media Capture can generate an Encoded Stream (Section 2.1.7) if o A media capture can generate an Encoded Stream (Section 2.1.7) if
the capture device supports such a configuration. the capture device supports such a configuration.
o The nature of the Media Capture may impose constraints on the o The nature of the media capture may impose constraints on the
clock handling in some of the subsequent steps. For example, many clock handling in some of the subsequent steps. For example, many
audio or video capture devices are not completely free in audio or video capture devices are not completely free in
selecting the sample rate. selecting the sample rate.
2.1.3. Raw Stream 2.1.3. Raw Stream
A Raw Stream is the time progressing stream of digitally sampled A raw stream is the time progressing stream of digitally sampled
information, usually periodically sampled and provided by a Media information, usually periodically sampled and provided by a media
Capture (Section 2.1.2). A Raw Stream can also contain synthesized capture (Section 2.1.2). A raw stream can also contain synthesized
Media that may not require any explicit Media Capture, since it is media that may not require any explicit media capture, since it is
already in an appropriate digital form. already in an appropriate digital form.
2.1.4. Media Source 2.1.4. Media Source
A Media Source is the logical source of a time progressing digital A Media Source is the logical source of a time progressing digital
media stream synchronized to a reference clock. This stream is media stream synchronized to a reference clock. This stream is
called a Source Stream (Section 2.1.5). This transformation takes called a source stream (Section 2.1.5). This transformation takes
one or more Raw Streams (Section 2.1.3) and provides a Source Stream one or more raw streams (Section 2.1.3) and provides a source stream
as output. The output is synchronized with a reference clock as output. The output is synchronized with a reference clock
(Section 3.1), which can be as simple as a system local wall clock or (Section 3.1), which can be as simple as a system local wall clock or
as complex as an NTP synchronized clock. as complex as an NTP synchronized clock.
The output can be of different types. One type is directly The output can be of different types. One type is directly
associated with a particular Media Capture's Raw Stream. Others are associated with a particular media capture's raw stream. Others are
more conceptual sources, like an audio mix of multiple Source Streams more conceptual sources, like an audio mix of multiple source streams
(Figure 3). Mixing multiple streams typically requires that the (Figure 3). Mixing multiple streams typically requires that the
input streams are possible to relate in time, meaning that they have input streams are possible to relate in time, meaning that they have
to be Source Streams (Section 2.1.5) rather than Raw Streams. In to be source streams (Section 2.1.5) rather than raw streams. In
Figure 3, the generated Source Stream is a mix of the three input Figure 3, the generated source stream is a mix of the three input
Source Streams. source streams.
Source Source Source Source Source Source
Stream Stream Stream Stream Stream Stream
| | | | | |
V V V V V V
+--------------------------+ +--------------------------+
| Media Source |<-- Reference Clock | Media Source |<-- Reference Clock
| Mixer | | Mixer |
+--------------------------+ +--------------------------+
| |
V V
Source Stream Source Stream
Figure 3: Conceptual Media Source in form of Audio Mixer Figure 3: Conceptual Media Source in the form of an Audio Mixer
Another possible example of a conceptual Media Source is a video Another possible example of a conceptual media source is a video
surveillance switch, where the input is multiple Source Streams from surveillance switch, where the input is multiple source streams from
different cameras, and the output is one of those Source Streams different cameras, and the output is one of those source streams
based on some selection criteria, like a round-robin or based on some based on some selection criteria, such as round robin or some video
video activity measure. activity measure.
2.1.5. Source Stream 2.1.5. Source Stream
A Source Stream is a stream of digital samples that has been A source stream is a stream of digital samples that has been
synchronized with a reference clock and comes from particular Media synchronized with a reference clock and comes from a particular media
Source (Section 2.1.4). source (Section 2.1.4).
2.1.6. Media Encoder 2.1.6. Media Encoder
A Media Encoder is a transform that is responsible for encoding the A media encoder is a transform that is responsible for encoding the
media data from a Source Stream (Section 2.1.5) into another media data from a source stream (Section 2.1.5) into another
representation, usually more compact, that is output as an Encoded representation, usually more compact, that is output as an encoded
Stream (Section 2.1.7). stream (Section 2.1.7).
The Media Encoder step commonly includes pre-encoding The media encoder step commonly includes pre-encoding
transformations, such as scaling, resampling etc. The Media Encoder transformations, such as scaling, resampling, etc. The media encoder
can have a significant number of configuration options that affects can have a significant number of configuration options that affects
the properties of the Encoded Stream. This include properties such the properties of the encoded stream. This includes properties such
as codec, bit-rate, start points for decoding, resolution, bandwidth as codec, bitrate, start points for decoding, resolution, bandwidth,
or other fidelity affecting properties. or other fidelity affecting properties.
Scalable Media Encoders need special attention as they produce Scalable media encoders need special attention as they produce
multiple outputs that are potentially of different types. As shown multiple outputs that are potentially of different types. As shown
in Figure 4, a scalable Media Encoder takes one input Source Stream in Figure 4, a scalable media encoder takes one input source stream
and encodes it into multiple output streams of two different types; and encodes it into multiple output streams of two different types:
at least one Encoded Stream that is independently decodable and one at least one encoded stream that is independently decodable and one
or more Dependent Streams (Section 2.1.8). Decoding requires at or more Dependent Streams (Section 2.1.8). Decoding requires at
least one Encoded Stream and zero or more Dependent Streams. A least one encoded stream and zero or more dependent streams. A
Dependent Stream's dependency is one of the grouping relations this dependent stream's dependency is one of the grouping relations this
document discusses further in Section 3.7. document discusses further in Section 3.7.
Source Stream Source Stream
| |
V V
+--------------------------+ +--------------------------+
| Scalable Media Encoder | | Scalable Media Encoder |
+--------------------------+ +--------------------------+
| | ... | | | ... |
V V V V V V
Encoded Dependent Dependent Encoded Dependent Dependent
Stream Stream Stream Stream Stream Stream
Figure 4: Scalable Media Encoder Input and Outputs Figure 4: Scalable Media Encoder Input and Outputs
There are also other variants of encoders, like so-called Multiple There are also other variants of encoders, like so-called Multiple
Description Coding (MDC). Such Media Encoders produce multiple Description Coding (MDC). Such media encoders produce multiple
independent and thus individually decodable Encoded Streams. independent and thus individually decodable encoded streams.
However, (logically) combining multiple of these Encoded Streams into However, (logically) combining multiple of these encoded streams into
a single Received Source Stream during decoding leads to an a single Received Source Stream during decoding leads to an
improvement in perceptual reproduced quality when compared to improvement in perceptual reproduced quality when compared to
decoding a single Encoded Stream. decoding a single encoded stream.
Creating multiple Encoded Streams from the same Source Stream, where Creating multiple encoded streams from the same source stream, where
the Encoded Streams are neither in a scalable nor in an MDC the encoded streams are neither in a scalable nor in an MDC
relationship is commonly utilized in Simulcast relationship is commonly utilized in simulcast [SDP-SIMULCAST]
[I-D.ietf-mmusic-sdp-simulcast] environments. environments.
2.1.7. Encoded Stream 2.1.7. Encoded Stream
A stream of time synchronized encoded media that can be independently A stream of time synchronized encoded media that can be independently
decoded. decoded.
Due to temporal dependencies, an Encoded Stream may have limitations Due to temporal dependencies, an encoded stream may have limitations
in where decoding can be started. These entry points, for example in where decoding can be started. These entry points, for example,
Intra frames from a video encoder, may require identification and Intra frames from a video encoder, may require identification and
their generation may be event based or configured to occur their generation may be event based or configured to occur
periodically. periodically.
2.1.8. Dependent Stream 2.1.8. Dependent Stream
A stream of time synchronized encoded media fragments that are A stream of time synchronized encoded media fragments that are
dependent on one or more Encoded Streams (Section 2.1.7) and zero or dependent on one or more encoded streams (Section 2.1.7) and zero or
more Dependent Streams to be possible to decode. more dependent streams to be possible to decode.
Each Dependent Stream has a set of dependencies. These dependencies Each dependent stream has a set of dependencies. These dependencies
must be understood by the parties in a Multimedia Session that intend must be understood by the parties in a Multimedia Session
to use a Dependent Stream. (Section 2.2.4) that intend to use a dependent stream.
2.1.9. Media Packetizer 2.1.9. Media Packetizer
The transformation of taking one or more Encoded (Section 2.1.7) or The transformation of taking one or more encoded (Section 2.1.7) or
Dependent Streams (Section 2.1.8) and putting their content into one dependent streams (Section 2.1.8) and putting their content into one
or more sequences of packets, normally RTP packets, and output Source or more sequences of packets, normally RTP Packets, and output Source
RTP Streams (Section 2.1.10). This step includes both generating RTP RTP Streams (Section 2.1.10). This step includes both generating RTP
payloads as well as RTP packets. The Media Packetizer then selects Payloads as well as RTP packets. The Media Packetizer then selects
which Synchronization source(s) (SSRC) [RFC3550] and RTP Sessions to which synchronization source(s) (SSRC) [RFC3550] and RTP Sessions
use. (Section 2.2.2) to use.
The Media Packetizer can combine multiple Encoded or Dependent The media packetizer can combine multiple encoded or dependent
Streams into one or more RTP Streams: streams into one or more RTP Streams:
o The Media Packetizer can use multiple inputs when producing a o The media packetizer can use multiple inputs when producing a
single RTP Stream. One such example is SRST packetization when single RTP stream. One such example is Single RTP stream on a
using Scalable Video Coding (SVC) (Section 3.7). Single media Transport (SRST) packetization when using Scalable
Video Coding (SVC) (Section 3.7).
o The Media Packetizer can also produce multiple RTP Streams, for o The media packetizer can also produce multiple RTP streams, for
example when Encoded and/or Dependent Streams are distributed over example, when encoded and/or dependent streams are distributed
multiple RTP Streams. One example of this is MRMT packetization over multiple RTP streams. One example of this is Multiple RTP
when using SVC (Section 3.7). streams on Multiple media Transports (MRMT) packetization when
using SVC (Section 3.7).
2.1.10. RTP Stream 2.1.10. RTP Stream
An RTP Stream is a stream of RTP packets containing media data, An RTP stream is a stream of RTP packets containing media data,
source or redundant. The RTP Stream is identified by an SSRC source or redundant. The RTP stream is identified by an SSRC
belonging to a particular RTP Session. The RTP Session is identified belonging to a particular RTP Session. The RTP session is identified
as discussed in Section 2.2.2. as discussed in Section 2.2.2.
A Source RTP Stream is an RTP Stream directly related to an Encoded A source RTP stream is an RTP stream directly related to an encoded
Stream (Section 2.1.7), targeted for transport over RTP without any stream (Section 2.1.7), targeted for transport over RTP without any
additional RTP-based Redundancy (Section 2.1.11) applied. additional RTP-based Redundancy (Section 2.1.11) applied.
Characteristics: Characteristics:
o Each RTP Stream is identified by a Synchronization source (SSRC) o Each RTP stream is identified by an SSRC [RFC3550] that is carried
[RFC3550] that is carried in every RTP and RTP Control Protocol in every RTP and RTP Control Protocol (RTCP) packet header. The
(RTCP) packet header. The SSRC is unique in a specific RTP SSRC is unique in a specific RTP session context.
Session context.
o At any given point in time, a RTP Stream can have one and only one o At any given point in time, an RTP stream can have one and only
SSRC, but SSRCs for a given RTP Stream can change over time. SSRC one SSRC, but SSRCs for a given RTP stream can change over time.
collision and clock rate change [RFC7160] are examples of valid SSRC collision and clock rate change [RFC7160] are examples of
reasons to change SSRC for an RTP Stream. In those cases, the RTP valid reasons to change SSRC for an RTP stream. In those cases,
Stream itself is not changed in any significant way, only the the RTP stream itself is not changed in any significant way, only
identifying SSRC number. the identifying SSRC number.
o Each SSRC defines a unique RTP sequence numbering and timing o Each SSRC defines a unique RTP sequence numbering and timing
space. space.
o Several RTP Streams, each with their own SSRC, may represent a o Several RTP streams, each with their own SSRC, may represent a
single Media Source. single media source.
o Several RTP Streams, each with their own SSRC, can be carried in a o Several RTP streams, each with their own SSRC, can be carried in a
single RTP Session. single RTP session.
2.1.11. RTP-based Redundancy 2.1.11. RTP-Based Redundancy
RTP-based Redundancy is defined here as a transformation that RTP-based redundancy is defined here as a transformation that
generates redundant or repair packets sent out as a Redundancy RTP generates redundant or repair packets sent out as a Redundancy RTP
Stream (Section 2.1.12) to mitigate network transport impairments, Stream (Section 2.1.12) to mitigate Network Transport
like packet loss and delay. Note that this excludes the type of (Section 2.1.18) impairments, like packet loss and delay. Note that
redundancy that most suitable Media Encoders (Section 2.1.6) may add this excludes the type of redundancy that most suitable media
to the media format of the Encoded Stream (Section 2.1.7) that makes encoders (Section 2.1.6) may add to the media format of the encoded
it cope better with inevitable RTP packet losses. stream (Section 2.1.7) that makes it cope better with RTP packet
losses.
The RTP-based Redundancy exists in many flavors; they may be The RTP-based redundancy exists in many flavors: they may generate
generating independent Repair Streams that are used in addition to independent repair streams that are used in addition to the source
the Source Stream (like RTP Retransmission (Section 3.10) and some stream (like RTP Retransmission (Section 3.10) and some special types
special types of Forward Error Correction, like RTP stream of Forward Error Correction (FEC) (Section 3.11), like RTP stream
duplication (Section 3.8)), they may generate a new Source Stream by duplication (Section 3.8)); they may generate a new source stream by
combining redundancy information with source information (Using XOR combining redundancy information with source information (using XOR
FEC (Section 3.11) as a redundancy payload (Section 3.9)), or FEC as a redundancy payload (Section 3.9)); or they may completely
completely replace the source information with only redundancy replace the source information with only redundancy packets.
packets.
2.1.12. Redundancy RTP Stream 2.1.12. Redundancy RTP Stream
A Redundancy RTP Stream is an RTP Stream (Section 2.1.10) that A redundancy RTP stream is an RTP stream (Section 2.1.10) that
contains no original source data, only redundant data, which may contains no original source data, only redundant data, which may
either be used standalone or be combined with one or more Received either be used as standalone or be combined with one or more Received
RTP Streams (Section 2.1.23) to produce Repaired RTP Streams RTP Streams (Section 2.1.23) to produce Repaired RTP Streams
(Section 2.1.26). (Section 2.1.26).
2.1.13. RTP-based Security 2.1.13. RTP-Based Security
The optional RTP-based Security transformation applies security The optional RTP-based Security transformation applies security
services such as authentication, integrity protection and services such as authentication, integrity protection, and
confidentiality to an input RTP Stream, like what is specified in The confidentiality to an input RTP stream, like what is specified in
Secure Real-time Transport Protocol (SRTP) [RFC3711], producing a "The Secure Real-time Transport Protocol (SRTP)" [RFC3711], producing
Secured RTP Stream (Section 2.1.14). Either an RTP Stream a Secured RTP Stream (Section 2.1.14). Either an RTP stream
(Section 2.1.10) or a Redundancy RTP Stream (Section 2.1.12) can be (Section 2.1.10) or a redundancy RTP stream (Section 2.1.12) can be
used as input to this transformation. used as input to this transformation.
In SRTP and the related Secure RTCP (SRTCP), all of the above In SRTP and the related Secure RTCP (SRTCP), all of the above-
mentioned security services are optional, except for integrity mentioned security services are optional, except for integrity
protection of SRTCP, which is mandatory. Also confidentiality protection of SRTCP, which is mandatory. Also confidentiality
(encryption) is effectively optional in SRTP, since it is possible to (encryption) is effectively optional in SRTP, since it is possible to
use a NULL encryption algorithm. As described in [RFC7201], the use a NULL encryption algorithm. As described in [RFC7201], the
strength of SRTP data origin authentication depends on the strength of SRTP data origin authentication depends on the
cryptographic transform and key management used, for example in group cryptographic transform and key management used. For example, in
communication where it is sometimes possible to authenticate group group communication, where it is sometimes possible to authenticate
membership but not the actual RTP Stream sender. group membership but not the actual RTP stream sender.
RTP-based Security and RTP-based Redundancy can be combined in a few RTP-based security and RTP-based redundancy can be combined in a few
different ways. One way is depicted in Figure 1, where an RTP Stream different ways. One way is depicted in Figure 1, where an RTP stream
and its corresponding Redundancy RTP Stream are protected by separate and its corresponding redundancy RTP stream are protected by separate
RTP-based Security transforms. In other cases, like when a Media RTP-based security transforms. In other cases, like when a Media
Translator is adding FEC in Section 3.2.1.3 of Translator is adding FEC in Section 3.2.1.3 of [RTP-TOPOLOGIES], a
[I-D.ietf-avtcore-rtp-topologies-update], a middlebox can apply RTP- middlebox can apply RTP-based redundancy to an already secured RTP
based Redundancy to an already Secured RTP Stream instead of a Source stream instead of a source RTP stream. One example of that is
RTP Stream. One example of that is depicted in Figure 5 below. depicted in Figure 5 below.
Source RTP Stream +------------+ Source RTP Stream +------------+
V | V V | V
+----------------------+ | +----------------------+ +----------------------+ | +----------------------+
| RTP-based Security | | | RTP-based Redundancy | | RTP-Based Security | | | RTP-Based Redundancy |
+----------------------+ | +----------------------+ +----------------------+ | +----------------------+
| | | | | |
| | Redundancy RTP Stream | | Redundancy RTP Stream
+-------------+ | +-------------+ |
| V | V
| +----------------------+ | +----------------------+
Secured RTP Stream | RTP-based Security | Secured RTP Stream | RTP-Based Security |
| +----------------------+ | +----------------------+
| | | |
| Secured Redundancy RTP Stream | Secured Redundancy RTP Stream
V V V V
+----------------------+ +----------------------+ +----------------------+ +----------------------+
| Media Transport | | Media Transport | | Media Transport | | Media Transport |
+----------------------+ +----------------------+ +----------------------+ +----------------------+
Figure 5: Adding Redundancy to a Secured RTP Stream Figure 5: Adding Redundancy to a Secured RTP Stream
In this case, the Redundancy RTP Stream may already have been secured In this case, the redundancy RTP stream may already have been secured
for confidentiality (encrypted) by the first RTP-based Security, and for confidentiality (encrypted) by the first RTP-based security, and
it may therefore not be necessary to apply additional confidentiality it may therefore not be necessary to apply additional confidentiality
protection in the second RTP-based Security. To avoid attacks and protection in the second RTP-based security. To avoid attacks and
negative impact on RTP-based Repair (Section 2.1.25) and the negative impact on RTP-based Repair (Section 2.1.25) and the
resulting Repaired RTP Stream (Section 2.1.26), it is however still resulting repaired RTP stream (Section 2.1.26), it is, however, still
necessary to have this second RTP-based Security apply both necessary to have this second RTP-based security apply both
authentication and integrity protection to the Redundancy RTP Stream. authentication and integrity protection to the redundancy RTP stream.
2.1.14. Secured RTP Stream 2.1.14. Secured RTP Stream
A Secured RTP Stream is a Source or Redundancy RTP Stream that is A secured RTP stream is a source or redundancy RTP stream that is
protected through RTP-based Security (Section 2.1.13) by one or more protected through RTP-based security (Section 2.1.13) by one or more
of the confidentiality, integrity, or authentication security of the confidentiality, integrity, or authentication security
services. services.
2.1.15. Media Transport 2.1.15. Media Transport
A Media Transport defines the transformation that the RTP Streams A media transport defines the transformation that the RTP streams
(Section 2.1.10) are subjected to by the end-to-end transport from (Section 2.1.10) are subjected to by the end-to-end transport from
one RTP sender to one specific RTP receiver (an RTP Session one RTP Sender (Section 4.12) to one specific RTP Receiver
(Section 2.2.2) may contain multiple RTP receivers per sender). Each (Section 4.11) (an RTP session (Section 2.2.2) may contain multiple
Media Transport is defined by a transport association that is RTP receivers per sender). Each media transport is defined by a
normally identified by a 5-tuple (source address, source port, transport association that is normally identified by a 5-tuple
destination address, destination port, transport protocol), but a (source address, source port, destination address, destination port,
proposal exists for sending multiple transport associations on a transport protocol), but a proposal exists for sending multiple
single 5-tuple [I-D.westerlund-avtcore-transport-multiplexing]. transport associations on a single 5-tuple [TRANSPORT-MULTIPLEX].
Characteristics: Characteristics:
o Media Transport transmits RTP Streams of RTP Packets from a source o Media transport transmits RTP streams of RTP packets from a source
transport address to a destination transport address. transport address to a destination transport address.
o Each Media Transport contains only a single RTP Session. o Each media transport contains only a single RTP session.
o A single RTP Session can span multiple Media Transports. o A single RTP session can span multiple media transports.
The Media Transport concept sometimes needs to be decomposed into The media transport concept sometimes needs to be decomposed into
more steps to enable discussion of what a sender emits that gets more steps to enable discussion of what a sender emits that gets
transformed by the network before it is received by the receiver. transformed by the network before it is received by the receiver.
Thus we provide also this Media Transport decomposition (Figure 6). Thus, we provide also this media transport decomposition (Figure 6).
RTP Stream RTP Stream
| |
V V
+--------------------------+ +--------------------------+
| Media Transport Sender | | Media Transport Sender |
+--------------------------+ +--------------------------+
| |
Sent RTP Stream Sent RTP Stream
V V
skipping to change at page 16, line 45 skipping to change at page 17, line 45
| Media Transport Receiver | | Media Transport Receiver |
+--------------------------+ +--------------------------+
| |
V V
Received RTP Stream Received RTP Stream
Figure 6: Decomposition of Media Transport Figure 6: Decomposition of Media Transport
2.1.16. Media Transport Sender 2.1.16. Media Transport Sender
The first transformation within the Media Transport (Section 2.1.15) The first transformation within the media transport (Section 2.1.15)
is the Media Transport Sender. The sending Endpoint (Section 2.2.1) is the Media Transport Sender. The sending Endpoint (Section 2.2.1)
takes an RTP Stream and emits the packets onto the network using the takes an RTP stream and emits the packets onto the network using the
transport association established for this Media Transport, thereby transport association established for this media transport, thereby
creating a Sent RTP Stream (Section 2.1.17). In the process, it creating a Sent RTP Stream (Section 2.1.17). In the process, it
transforms the RTP Stream in several ways. First, it generates the transforms the RTP stream in several ways. First, it generates the
necessary protocol headers for the transport association, for example necessary protocol headers for the transport association, for
IP and UDP headers, thus forming IP/UDP/RTP packets. In addition, example, IP and UDP headers, thus forming IP/UDP/RTP packets. In
the Media Transport Sender may queue, intentionally pace or otherwise addition, the media transport sender may queue, intentionally pace,
affect how the packets are emitted onto the network, thereby or otherwise affect how the packets are emitted onto the network,
potentially introducing delay and delay variations [RFC5481] that thereby potentially introducing delay and delay variations [RFC5481]
characterize the Sent RTP Stream. that characterize the sent RTP stream.
2.1.17. Sent RTP Stream 2.1.17. Sent RTP Stream
The Sent RTP Stream is the RTP Stream as entering the first hop of The sent RTP stream is the RTP stream as entering the first hop of
the network path to its destination. The Sent RTP Stream is the network path to its destination. The sent RTP stream is
identified using network transport addresses, like for IP/UDP the identified using network transport addresses, like the 5-tuple
5-tuple (source IP address, source port, destination IP address, (source IP address, source port, destination IP address, destination
destination port, and protocol (UDP)). port, and protocol (UDP)) for IP/UDP.
2.1.18. Network Transport 2.1.18. Network Transport
Network Transport is the transformation that subjects the Sent RTP Network transport is the transformation that subjects the sent RTP
Stream (Section 2.1.17) to traveling from the source to the stream (Section 2.1.17) to traveling from the source to the
destination through the network. This transformation can result in destination through the network. This transformation can result in
loss of some packets, delay and delay variation on a per packet loss of some packets, delay, and delay variation on a per-packet
basis, packet duplication, and packet header or data corruption. basis, packet duplication, and packet header or data corruption.
This transformation produces a Transported RTP Stream This transformation produces a Transported RTP Stream
(Section 2.1.19) at the exit of the network path. (Section 2.1.19) at the exit of the network path.
2.1.19. Transported RTP Stream 2.1.19. Transported RTP Stream
The Transported RTP Stream is the RTP Stream that is emitted out of The transported RTP stream is the RTP stream that is emitted out of
the network path at the destination, subjected to the Network the network path at the destination, subjected to the network
Transport's transformation (Section 2.1.18). transport's transformation (Section 2.1.18).
2.1.20. Media Transport Receiver 2.1.20. Media Transport Receiver
The Media Transport Receiver is the receiver Endpoint's The Media Transport Receiver is the receiver endpoint's
(Section 2.2.1) transformation of the Transported RTP Stream (Section 2.2.1) transformation of the transported RTP stream
(Section 2.1.19) by its reception process, which results in the (Section 2.1.19) by its reception process, which results in the
Received RTP Stream (Section 2.1.23). This transformation includes received RTP stream (Section 2.1.23). This transformation includes
transport checksums being verified. Sensible system designs transport checksums being verified. Sensible system designs
typically either discard packets with mis-matching checksums, or pass typically either discard packets with mismatching checksums or pass
them on while somehow marking them in the resulting Received RTP them on while somehow marking them in the resulting received RTP
Stream so to alert subsequent transformations about the possible stream so to alert subsequent transformations about the possible
corrupt state. In this context it is worth noting that there is corrupt state. In this context, it is worth noting that there is
typically some probability for corrupt packets to pass through typically some probability for corrupt packets to pass through
undetected (with a seemingly correct checksum). Other undetected (with a seemingly correct checksum). Other
transformations can compensate for delay variations in receiving a transformations can compensate for delay variations in receiving a
packet on the network interface and providing it to the application packet on the network interface and providing it to the application
(de-jitter buffer). (de-jitter buffer).
2.1.21. Received Secured RTP Stream 2.1.21. Received Secured RTP Stream
This is the Secured RTP Stream (Section 2.1.14) resulting from the This is the secured RTP stream (Section 2.1.14) resulting from the
Media Transport (Section 2.1.15) aggregate transformation. media transport (Section 2.1.15) aggregate transformation.
2.1.22. RTP-based Validation 2.1.22. RTP-Based Validation
RTP-based Validation is the reverse transformation of RTP-based RTP-based Validation is the reverse transformation of RTP-based
Security (Section 2.1.13). If this transformation fails, the result security (Section 2.1.13). If this transformation fails, the result
is either not usable and must be discarded, or may be usable but is either not usable and must be discarded or may be usable but
cannot be trusted. If the transformation succeeds, the result can be cannot be trusted. If the transformation succeeds, the result can be
a Received RTP Stream (Section 2.1.23) or a Received Redundancy RTP a received RTP stream (Section 2.1.23) or a Received Redundancy RTP
Stream (Section 2.1.24), depending on what was input to the Stream (Section 2.1.24), depending on what was input to the
corresponding RTP-based Security transformation, but can also be a corresponding RTP-based security transformation, but it can also be a
Received Secured RTP Stream (Section 2.1.21) in case several RTP- Received Secured RTP Stream (Section 2.1.21) in case several RTP-
based Security transformations were applied. based security transformations were applied.
2.1.23. Received RTP Stream 2.1.23. Received RTP Stream
The Received RTP Stream is the RTP Stream (Section 2.1.10) resulting The received RTP stream is the RTP stream (Section 2.1.10) resulting
from the Media Transport's aggregate transformation (Section 2.1.15), from the media transport's aggregate transformation (Section 2.1.15),
i.e. subjected to packet loss, packet corruption, packet duplication, i.e., subjected to packet loss, packet corruption, packet
delay, and delay variation from sender to receiver. duplication, delay, and delay variation from sender to receiver.
2.1.24. Received Redundancy RTP Stream 2.1.24. Received Redundancy RTP Stream
The Received Redundancy RTP Stream is the Redundancy RTP Stream The received redundancy RTP stream is the redundancy RTP stream
(Section 2.1.12) resulting from the Media Transport transformation, (Section 2.1.12) resulting from the media transport's aggregate
i.e. subjected to packet loss, packet corruption, delay, and delay transformation, i.e., subjected to packet loss, packet corruption,
variation from sender to receiver. packet duplication, delay, and delay variation from sender to
receiver.
2.1.25. RTP-based Repair 2.1.25. RTP-Based Repair
RTP-based Repair is a Transformation that takes as input zero or more RTP-based repair is a transformation that takes as input zero or more
Received RTP Streams (Section 2.1.23) and one or more Received received RTP streams (Section 2.1.23) and one or more received
Redundancy RTP Streams (Section 2.1.24), and produces one or more redundancy RTP streams (Section 2.1.24) and produces one or more
Repaired RTP Streams (Section 2.1.26) that are as close to the repaired RTP streams (Section 2.1.26) that are as close to the
corresponding sent Source RTP Streams (Section 2.1.10) as possible, corresponding sent source RTP streams (Section 2.1.10) as possible,
using different RTP-based repair methods, for example the ones using different RTP-based repair methods, for example, the ones
referred in RTP-based Redundancy (Section 2.1.11). referred to in RTP-based redundancy (Section 2.1.11).
2.1.26. Repaired RTP Stream 2.1.26. Repaired RTP Stream
A Repaired RTP Stream is a Received RTP Stream (Section 2.1.23) for A repaired RTP stream is a received RTP stream (Section 2.1.23) for
which Received Redundancy RTP Stream (Section 2.1.24) information has which received redundancy RTP stream (Section 2.1.24) information has
been used to try to recover the Source RTP Stream (Section 2.1.10) as been used to try to recover the source RTP stream (Section 2.1.10) as
it was before Media Transport (Section 2.1.15). it was before media transport (Section 2.1.15).
2.1.27. Media Depacketizer 2.1.27. Media Depacketizer
A Media Depacketizer takes one or more RTP Streams (Section 2.1.10), A Media Depacketizer takes one or more RTP streams (Section 2.1.10),
depacketizes them, and attempts to reconstitute the Encoded Streams depacketizes them, and attempts to reconstitute the encoded streams
(Section 2.1.7) or Dependent Streams (Section 2.1.8) present in those (Section 2.1.7) or dependent streams (Section 2.1.8) present in those
RTP Streams. RTP streams.
In practical implementations, the Media Depacketizer and the Media In practical implementations, the media depacketizer and the media
Decoder may be tightly coupled and share information to improve or decoder may be tightly coupled and share information to improve or
optimize the overall decoding and error concealment process. It is, optimize the overall decoding and error concealment process. It is,
however, not expected that there would be any benefit in defining a however, not expected that there would be any benefit in defining a
taxonomy for those detailed (and likely very implementation- taxonomy for those detailed (and likely very implementation-
dependent) steps. dependent) steps.
2.1.28. Received Encoded Stream 2.1.28. Received Encoded Stream
The Received Encoded Stream is the received version of an Encoded The Received Encoded Stream is the received version of an encoded
Stream (Section 2.1.7). stream (Section 2.1.7).
2.1.29. Media Decoder 2.1.29. Media Decoder
A Media Decoder is a transformation that is responsible for decoding A media decoder is a transformation that is responsible for decoding
Encoded Streams (Section 2.1.7) and any Dependent Streams encoded streams (Section 2.1.7) and any dependent streams
(Section 2.1.8) into a Source Stream (Section 2.1.5). (Section 2.1.8) into a source stream (Section 2.1.5).
In practical implementations, the Media Decoder and the Media In practical implementations, the media decoder and the media
Depacketizer may be tightly coupled and share information to improve depacketizer may be tightly coupled and share information to improve
or optimize the overall decoding process in various ways. It is or optimize the overall decoding process in various ways. It is,
however not expected that there would be any benefit in defining a however, not expected that there would be any benefit in defining a
taxonomy for those detailed (and likely very implementation- taxonomy for those detailed (and likely very implementation-
dependent) steps. dependent) steps.
A Media Decoder has to deal with any errors in the Encoded Streams A media decoder has to deal with any errors in the encoded streams
that resulted from corruption or failure to repair packet losses. that resulted from corruption or failure to repair packet losses.
Therefore, it commonly is robust to error and losses, and includes Therefore, it commonly is robust to error and losses, and includes
concealment methods. concealment methods.
2.1.30. Received Source Stream 2.1.30. Received Source Stream
The Received Source Stream is the received version of a Source Stream The received source stream is the received version of a source stream
(Section 2.1.5). (Section 2.1.5).
2.1.31. Media Sink 2.1.31. Media Sink
The Media Sink receives a Source Stream (Section 2.1.5) that The Media Sink receives a source stream (Section 2.1.5) that
contains, usually periodically, sampled media data together with contains, usually periodically, sampled media data together with
associated synchronization information. Depending on application, associated synchronization information. Depending on application,
this Source Stream then needs to be transformed into a Raw Stream this source stream then needs to be transformed into a raw stream
(Section 2.1.3) that is conveyed to the Media Render (Section 2.1.3) that is conveyed to the Media Render (Section 2.1.33)
(Section 2.1.33), synchronized with the output from other Media and synchronized with the output from other media sinks. The media
Sinks. The Media Sink may also be connected with a Media Source sink may also be connected with a media source (Section 2.1.4) and be
(Section 2.1.4) and be used as part of a conceptual Media Source. used as part of a conceptual media source.
The Media Sink can further transform the Source Stream into a The media sink can further transform the source stream into a
representation that is suitable for rendering on the Media Render as representation that is suitable for rendering on the media render as
defined by the application or system-wide configuration. This defined by the application or system-wide configuration. This
include sample scaling, level adjustments etc. includes sample scaling, level adjustments, etc.
2.1.32. Received Raw Stream 2.1.32. Received Raw Stream
The Received Raw Stream is the received version of a Raw Stream The Received Raw Stream is the received version of a raw stream
(Section 2.1.3). (Section 2.1.3).
2.1.33. Media Render 2.1.33. Media Render
A Media Render takes a Raw Stream (Section 2.1.3) and converts it A media render takes a raw stream (Section 2.1.3) and converts it
into Physical Stimulus (Section 2.1.1) that a human user can into physical stimulus (Section 2.1.1) that a human user can
perceive. Examples of such devices are screens, and D/A converters perceive. Examples of such devices are screens and D/A converters
connected to amplifiers and loudspeakers. connected to amplifiers and loudspeakers.
An Endpoint can potentially have multiple Media Renders for each An endpoint can potentially have multiple media renders for each
media type. media type.
2.2. Communication Entities 2.2. Communication Entities
This section contains concepts for entities involved in the This section contains concepts for entities involved in the
communication. communication.
+------------------------------------------------------------+ +------------------------------------------------------------+
| Communication Session | | Communication Session |
| | | |
skipping to change at page 21, line 36 skipping to change at page 22, line 41
| | | +----------+-+----------|-----------+-+----------+ | | | | | | +----------+-+----------|-----------+-+----------+ | | |
| | | | RTP | | v | | | | | | | | | | RTP | | v | | | | | |
| | | | Session |<+---Media Transport----+-| | | | | | | | | Session |<+---Media Transport----+-| | | | |
| | | | Video |-+---Media Transport----+>| | | | | | | | | Video |-+---Media Transport----+>| | | | |
| | | | | | | | | | | | | | | | | | | | | | | |
| | | +----------+-+----------------------+-+----------+ | | | | | | +----------+-+----------------------+-+----------+ | | |
| | +------------+ | | +------------+ | | | | +------------+ | | +------------+ | |
| +----------------+ +----------------+ | | +----------------+ +----------------+ |
+------------------------------------------------------------+ +------------------------------------------------------------+
Figure 7: Example Point to Point Communication Session with two RTP Figure 7: Example Point-to-Point Communication Session with Two RTP
Sessions Sessions
Figure 7 shows a high-level example representation of a very basic Figure 7 shows a high-level example representation of a very basic
point-to-point Communication Session between Participants A and B. point-to-point Communication Session between Participants A and B.
It uses two different audio and video RTP Sessions between A's and It uses two different audio and video RTP sessions between A's and
B's Endpoints, where each RTP Session is a group communications B's endpoints, where each RTP session is a group communications
channel that can potentially carry a number of RTP Streams. It is channel that can potentially carry a number of RTP streams. It is
using separate Media Transports for those RTP Sessions. The using separate media transports for those RTP sessions. The
Multimedia Session shared by the Participants can, for example, be multimedia session shared by the participants can, for example, be
established using SIP (i.e., there is a SIP Dialog between A and B). established using SIP (i.e., there is a SIP dialog between A and B).
The terms used in Figure 7 are further elaborated in the sub-sections
The terms used in Figure 7 are further elaborated in the subsections
below. below.
2.2.1. Endpoint 2.2.1. Endpoint
An Endpoint is a single addressable entity sending or receiving RTP An endpoint is a single addressable entity sending or receiving RTP
packets. It may be decomposed into several functional blocks, but as packets. It may be decomposed into several functional blocks, but as
long as it behaves as a single RTP stack entity it is classified as a long as it behaves as a single RTP stack entity, it is classified as
single "Endpoint". a single "endpoint".
Characteristics: Characteristics:
o Endpoints can be identified in several different ways. While RTCP o Endpoints can be identified in several different ways. While RTCP
Canonical Names (CNAMEs) [RFC3550] provide a globally unique and Canonical Names (CNAMEs) [RFC3550] provide a globally unique and
stable identification mechanism for the duration of the stable identification mechanism for the duration of the
Communication Session (see Section 2.2.5), their validity applies communication session (see Section 2.2.5), their validity applies
exclusively within a Synchronization Context (Section 3.1). Thus exclusively within a Synchronization Context (Section 3.1). Thus,
one Endpoint can handle multiple CNAMEs, each of which can be one endpoint can handle multiple CNAMEs, each of which can be
shared among a set of Endpoints belonging to the same Participant shared among a set of endpoints belonging to the same participant
(Section 2.2.3). Therefore, mechanisms outside the scope of RTP, (Section 2.2.3). Therefore, mechanisms outside the scope of RTP,
such as application defined mechanisms, must be used to provide such as application-defined mechanisms, must be used to provide
Endpoint identification when outside this Synchronization Context. endpoint identification when outside this synchronization context.
o An Endpoint can be associated with at most one Participant o An endpoint can be associated with at most one participant
(Section 2.2.3) at any single point in time. (Section 2.2.3) at any single point in time.
o In some contexts, an Endpoint would typically correspond to a o In some contexts, an endpoint would typically correspond to a
single "host", for example a computer using a single network single "host", for example, a computer using a single network
interface and being used by a single human user. In other interface and being used by a single human user. In other
contexts, a single "host" can serve multiple Participants, in contexts, a single "host" can serve multiple participants, in
which case each Participant's Endpoint may share properties, for which case each participant's endpoint may share properties, for
example the IP address part of a transport address. example, the IP address part of a transport address.
2.2.2. RTP Session 2.2.2. RTP Session
An RTP Session is an association among a group of Participants An RTP session is an association among a group of participants
communicating with RTP. It is a group communications channel which communicating with RTP. It is a group communications channel that
can potentially carry a number of RTP Streams. Within an RTP can potentially carry a number of RTP streams. Within an RTP
Session, every Participant can find meta-data and control information session, every participant can find metadata and control information
(over RTCP) about all the RTP Streams in the RTP Session. The (over RTCP) about all the RTP streams in the RTP session. The
bandwidth of the RTCP control channel is shared between all bandwidth of the RTCP control channel is shared between all
Participants within an RTP Session. participants within an RTP session.
Characteristics: Characteristics:
o An RTP Session can carry one ore more RTP Streams. o An RTP session can carry one or more RTP streams.
o An RTP Session shares a single SSRC space as defined in RFC3550 o An RTP session shares a single SSRC space as defined in [RFC3550].
[RFC3550]. That is, the Endpoints participating in an RTP Session That is, the endpoints participating in an RTP session can see an
can see an SSRC identifier transmitted by any of the other SSRC identifier transmitted by any of the other endpoints. An
Endpoints. An Endpoint can receive an SSRC either as SSRC or as a endpoint can receive an SSRC either as SSRC or as a contributing
Contributing source (CSRC) in RTP and RTCP packets, as defined by source (CSRC) in RTP and RTCP packets, as defined by the
the Endpoints' network interconnection topology. endpoints' network interconnection topology.
o An RTP Session uses at least two Media Transports o An RTP session uses at least two media transports
(Section 2.1.15), one for sending and one for receiving. (Section 2.1.15): one for sending and one for receiving.
Commonly, the receiving Media Transport is the reverse direction Commonly, the receiving media transport is the reverse direction
of the Media Transport used for sending. An RTP Session may use of the media transport used for sending. An RTP session may use
many Media Transports and these define the session's network many media transports and these define the session's network
interconnection topology. interconnection topology.
o A single Media Transport always carries a single RTP Session. o A single media transport always carries a single RTP session.
o Multiple RTP Sessions can be conceptually related, for example o Multiple RTP sessions can be conceptually related, for example,
originating from or targeted for the same Participant originating from or targeted for the same participant
(Section 2.2.3) or Endpoint (Section 2.2.1), or by containing RTP (Section 2.2.3) or endpoint (Section 2.2.1), or by containing RTP
Streams that are somehow related (Section 3). streams that are somehow related (Section 3).
2.2.3. Participant 2.2.3. Participant
A Participant is an entity reachable by a single signaling address, A participant is an entity reachable by a single signaling address
and is thus related more to the signaling context than to the media and is thus related more to the signaling context than to the media
context. context.
Characteristics: Characteristics:
o A single signaling-addressable entity, using an application- o A single signaling-addressable entity, using an application-
specific signaling address space, for example a SIP URI. specific signaling address space, for example, a SIP URI.
o A Participant can participate in several Multimedia Sessions o A participant can participate in several multimedia sessions
(Section 2.2.4). (Section 2.2.4).
o A Participant can be comprised of several associated Endpoints o A participant can be comprised of several associated endpoints
(Section 2.2.1). (Section 2.2.1).
2.2.4. Multimedia Session 2.2.4. Multimedia Session
A Multimedia Session is an association among a group of Participants A multimedia session is an association among a group of participants
(Section 2.2.3) engaged in the communication via one or more RTP (Section 2.2.3) engaged in the communication via one or more RTP
Sessions (Section 2.2.2). It defines logical relationships among sessions (Section 2.2.2). It defines logical relationships among
Media Sources (Section 2.1.4) that appear in multiple RTP Sessions. media sources (Section 2.1.4) that appear in multiple RTP sessions.
Characteristics: Characteristics:
o A Multimedia Session can be composed of several RTP Sessions with o A multimedia session can be composed of several RTP sessions with
potentially multiple RTP Streams per RTP Session. potentially multiple RTP streams per RTP session.
o Each Participant in a Multimedia Session can have a multitude of o Each participant in a multimedia session can have a multitude of
Media Captures and Media Rendering devices. media captures and media rendering devices.
o A single Multimedia Session can contain media from one or more o A single multimedia session can contain media from one or more
Synchronization Contexts (Section 3.1). An example of that is a synchronization contexts (Section 3.1). An example of that is a
Multimedia Session containing one set of audio and video for multimedia session containing one set of audio and video for
communication purposes belonging to one Synchronization Context, communication purposes belonging to one synchronization context,
and another set of audio and video for presentation purposes (like and another set of audio and video for presentation purposes (like
playing a video file) with a separate Synchronization Context that playing a video file) with a separate synchronization context that
has no strong timing relationship and need not be strictly has no strong timing relationship and need not be strictly
synchronized with the audio and video used for communication. synchronized with the audio and video used for communication.
2.2.5. Communication Session 2.2.5. Communication Session
A Communication Session is an association among two or more A communication session is an association among two or more
Participants (Section 2.2.3) communicating with each other via one or participants (Section 2.2.3) communicating with each other via one or
more Multimedia Sessions (Section 2.2.4). more multimedia sessions (Section 2.2.4).
Characteristics: Characteristics:
o Each Participant in a Communication Session is identified via an o Each participant in a communication session is identified via an
application-specific signaling address. application-specific signaling address.
o A Communication Session is composed of Participants that share at o A communication session is composed of participants that share at
least one Multimedia Session, involving one or more parallel RTP least one multimedia session, involving one or more parallel RTP
Sessions with potentially multiple RTP Streams per RTP Session. sessions with potentially multiple RTP streams per RTP session.
For example, in a full mesh communication, the Communication Session For example, in a full mesh communication, the communication session
consists of a set of separate Multimedia Sessions between each pair consists of a set of separate multimedia sessions between each pair
of Participants. Another example is a centralized conference, where of participants. Another example is a centralized conference, where
the Communication Session consists of a set of Multimedia Sessions the communication session consists of a set of multimedia sessions
between each Participant and the conference handler. between each participant and the conference handler.
3. Concepts of Inter-Relations 3. Concepts of Inter-Relations
This section uses the concepts from previous sections, and looks at This section uses the concepts from previous sections and looks at
different types of relationships among them. These relationships different types of relationships among them. These relationships
occur at different abstraction levels and for different purposes, but occur at different abstraction levels and for different purposes, but
the reason for the needed relationship at a certain step in the media the reason for the needed relationship at a certain step in the media
handling chain may exist at another step. For example, the use of handling chain may exist at another step. For example, the use of
Simulcast (Section 3.6)) implies a need to determine relations at RTP simulcast (Section 3.6) implies a need to determine relations at the
Stream level, but the underlying reason is that multiple Media RTP stream level, but the underlying reason is that multiple media
Encoders use the same Media Source, i.e. to be able to identify a encoders use the same media source, i.e., to be able to identify a
common Media Source. common media source.
3.1. Synchronization Context 3.1. Synchronization Context
A Synchronization Context defines a requirement on a strong timing A synchronization context defines a requirement for a strong timing
relationship between the Media Sources, typically requiring alignment relationship between the media sources, typically requiring alignment
of clock sources. Such a relationship can be identified in multiple of clock sources. Such a relationship can be identified in multiple
ways as listed below. A single Media Source can only belong to a ways as listed below. A single media source can only belong to a
single Synchronization Context, since it is assumed that a single single synchronization context, since it is assumed that a single
Media Source can only have a single media clock and requiring media source can only have a single media clock and requiring
alignment to several Synchronization Contexts (and thus reference alignment to several synchronization contexts (and thus reference
clocks) will effectively merge those into a single Synchronization clocks) will effectively merge those into a single synchronization
Context. context.
3.1.1. RTCP CNAME 3.1.1. RTCP CNAME
RFC3550 [RFC3550] describes Inter-media synchronization between RTP [RFC3550] describes inter-media synchronization between RTP sessions
Sessions based on RTCP CNAME, RTP and Network Time Protocol (NTP) based on RTCP CNAME, RTP, and timestamps of a reference clock
[RFC5905] formatted timestamps of a reference clock. As indicated in formatted using the Network Time Protocol (NTP) [RFC5905]. As
[RFC7273], despite using NTP format timestamps, it is not required indicated in [RFC7273], despite using NTP format timestamps, it is
that the clock be synchronized to an NTP source. not required that the clock be synchronized to an NTP source.
3.1.2. Clock Source Signaling 3.1.2. Clock Source Signaling
[RFC7273] provides a mechanism to signal the clock source in Session [RFC7273] provides a mechanism to signal the clock source in the
Description Protocol (SDP) [RFC4566] both for the reference clock as Session Description Protocol (SDP) [RFC4566] both for the reference
well as the media clock, thus allowing a Synchronization Context to clock as well as the media clock, thus allowing a synchronization
be defined beyond the one defined by the usage of CNAME source context to be defined beyond the one defined by the usage of CNAME
descriptions. source descriptions.
3.1.3. Implicitly via RtcMediaStream 3.1.3. Implicitly via RtcMediaStream
WebRTC defines "RtcMediaStream" with one or more WebRTC defines RtcMediaStream with one or more RtcMediaStreamTracks.
"RtcMediaStreamTracks". All tracks in a "RtcMediaStream" are All tracks in a RtcMediaStream are intended to be synchronized when
intended to be synchronized when rendered, implying that they must be rendered, implying that they must be generated such that
generated such that synchronization is possible. synchronization is possible.
3.1.4. Explicitly via SDP Mechanisms 3.1.4. Explicitly via SDP Mechanisms
The SDP Grouping Framework [RFC5888] defines an m= line (Section 4.2) The SDP Grouping Framework [RFC5888] defines an "m=" line
grouping mechanism called "Lip Synchronization" (with LS (Section 4.2) grouping mechanism called Lip Synchronization (with LS
identification-tag) for establishing the synchronization requirement identification-tag) for establishing the synchronization requirement
across m= lines when they map to individual sources. across "m=" lines when they map to individual sources.
Source-Specific Media Attributes in SDP [RFC5576] extends the above Source-Specific Media Attributes in SDP [RFC5576] extends the above
mechanism when multiple Media Sources are described by a single m= mechanism when multiple media sources are described by a single "m="
line. line.
3.2. Endpoint 3.2. Endpoint
Some applications requires knowledge of what Media Sources originate Some applications require knowledge of what media sources originate
from a particular Endpoint (Section 2.2.1). This can include such from a particular endpoint (Section 2.2.1). This can include such
decisions as packet routing between parts of the topology, knowing decisions as packet routing between parts of the topology, knowing
the Endpoint origin of the RTP Streams. the endpoint origin of the RTP streams.
In RTP, this identification has been overloaded with the In RTP, this identification has been overloaded with the
Synchronization Context (Section 3.1) through the usage of the RTCP synchronization context (Section 3.1) through the usage of the RTCP
source description CNAME (Section 3.1.1). This works for some source description CNAME (Section 3.1.1). This works for some
usages, but in others it breaks down. For example, if an Endpoint usages, but in others it breaks down. For example, if an endpoint
has two sets of Media Sources that have different Synchronization has two sets of media sources that have different synchronization
Contexts, like the audio and video of the human Participant as well contexts, like the audio and video of the human participant as well
as a set of Media Sources of audio and video for a shared movie, as a set of media sources of audio and video for a shared movie,
CNAME would not be an appropriate identification for that Endpoint. CNAME would not be an appropriate identification for that endpoint.
Therefore, an Endpoint may have multiple CNAMEs. The CNAMEs or the Therefore, an endpoint may have multiple CNAMEs. The CNAMEs or the
Media Sources themselves can be related to the Endpoint. media sources themselves can be related to the endpoint.
3.3. Participant 3.3. Participant
In communication scenarios, it is commonly needed to know which Media In communication scenarios, information about which media sources
Sources originate from which Participant (Section 2.2.3). One reason originate from which participant (Section 2.2.3) is commonly needed.
is, for example, to enable the application to display Participant One reason is, for example, to enable the application to correctly
Identity information correctly associated with the Media Sources. display participant identity information associated with the media
This association is handled through the signaling solution to point sources. This association is handled through signaling to point at a
at a specific Multimedia Session where the Media Sources may be specific multimedia session where the media sources may be explicitly
explicitly or implicitly tied to a particular Endpoint. or implicitly tied to a particular endpoint.
Participant information becomes more problematic due to Media Sources Participant information becomes more problematic when there are media
that are generated through mixing or other conceptual processing of sources that are generated through mixing or other conceptual
Raw Streams or Source Streams that originate from different processing of raw streams or source streams that originate from
Participants. This type of Media Sources can thus have a dynamically different participants. These types of media sources can thus have a
varying set of origins and Participants. RTP contains the concept of dynamically varying set of origins and participants. RTP contains
CSRC that carry information about the previous step origin of the the concept of CSRC that carries information about the previous step
included media content on RTP level. origin of the included media content on the RTP level.
3.4. RtcMediaStream 3.4. RtcMediaStream
An RtcMediaStream in WebRTC is an explicit grouping of a set of Media An RtcMediaStream in WebRTC is an explicit grouping of a set of media
Sources (RtcMediaStreamTracks) that share a common identifier and a sources (RtcMediaStreamTracks) that share a common identifier and a
single Synchronization Context (Section 3.1). single synchronization context (Section 3.1).
3.5. Multi-Channel Audio 3.5. Multi-Channel Audio
There exist a number of RTP payload formats that can carry multi- There exist a number of RTP payload formats that can carry multi-
channel audio, despite the codec being a single-channel (mono) channel audio, despite the codec being a single-channel (mono)
encoder. Multi-channel audio can be viewed as multiple Media Sources encoder. Multi-channel audio can be viewed as multiple media sources
sharing a common Synchronization Context. These are independently sharing a common synchronization context. These are independently
encoded by a Media Encoder and the different Encoded Streams are encoded by a media encoder and the different encoded streams are
packetized together in a time synchronized way into a single Source packetized together in a time-synchronized way into a single source
RTP Stream, using the used codec's RTP Payload format. Examples of RTP stream, using the used codec's RTP payload format. Examples of
codecs that support multi-channel audio are PCMA and PCMU [RFC3551], codecs that support multi-channel audio are PCMA and PCMU [RFC3551],
AMR [RFC4867], and G.719 [RFC5404]. Adaptive Multi Rate (AMR) [RFC4867], and G.719 [RFC5404].
3.6. Simulcast 3.6. Simulcast
A Media Source represented as multiple independent Encoded Streams A media source represented as multiple independent encoded streams
constitutes a Simulcast [I-D.ietf-mmusic-sdp-simulcast] or MDC of constitutes a simulcast [SDP-SIMULCAST] or Modification Detection
that Media Source. Figure 8 shows an example of a Media Source that Code (MDC) of that media source. Figure 8 shows an example of a
is encoded into three separate Simulcast streams, that are in turn media source that is encoded into three separate simulcast streams,
sent on the same Media Transport flow. When using Simulcast, the RTP that are in turn sent on the same media transport flow. When using
Streams may be sharing RTP Session and Media Transport, or be simulcast, the RTP streams may be sharing an RTP session and media
separated on different RTP Sessions and Media Transports, or any transport, or be separated on different RTP sessions and media
combination of these two. One major reason to use separate Media transports, or be any combination of these two. One major reason to
Transports is to make use of different Quality of Service for the use separate media transports is to make use of different quality of
different Source RTP Streams. Some considerations on separating service (QoS) for the different source RTP streams. Some
related RTP Streams are discussed in Section 3.12. considerations on separating related RTP streams are discussed in
Section 3.12.
+----------------+ +----------------+
| Media Source | | Media Source |
+----------------+ +----------------+
Source Stream | Source Stream |
+----------------------+----------------------+ +----------------------+----------------------+
| | | | | |
V V V V V V
+------------------+ +------------------+ +------------------+ +------------------+ +------------------+ +------------------+
| Media Encoder | | Media Encoder | | Media Encoder | | Media Encoder | | Media Encoder | | Media Encoder |
skipping to change at page 27, line 47 skipping to change at page 29, line 33
| Stream | Stream | Stream | Stream | Stream | Stream
+-----------------+ | +-----------------+ +-----------------+ | +-----------------+
| | | | | |
V V V V V V
+-------------------+ +-------------------+
| Media Transport | | Media Transport |
+-------------------+ +-------------------+
Figure 8: Example of Media Source Simulcast Figure 8: Example of Media Source Simulcast
The Simulcast relation between the RTP Streams is the common Media The simulcast relation between the RTP streams is the common media
Source. In addition, to be able to identify the common Media Source, source. In addition, to be able to identify the common media source,
a receiver of the RTP Stream may need to know which configuration or a receiver of the RTP stream may need to know which configuration or
encoding goals that lay behind the produced Encoded Stream and its encoding goals lay behind the produced encoded stream and its
properties. This enables selection of the stream that is most useful properties. This enables selection of the stream that is most useful
in the application at that moment. in the application at that moment.
3.7. Layered Multi-Stream 3.7. Layered Multi-Stream
Layered Multi-Stream (LMS) is a mechanism by which different portions Layered Multi-Stream (LMS) is a mechanism by which different portions
of a layered or scalable encoding of a Source Stream are sent using of a layered or scalable encoding of a source stream are sent using
separate RTP Streams (sometimes in separate RTP Sessions). LMSs are separate RTP streams (sometimes in separate RTP sessions). LMSs are
useful for receiver control of layered media. useful for receiver control of layered media.
A Media Source represented as an Encoded Stream and multiple A media source represented as an encoded stream and multiple
Dependent Streams constitutes a Media Source that has layered dependent streams constitutes a media source that has layered
dependencies. Figure 9 represents an example of a Media Source that dependencies. Figure 9 represents an example of a media source that
is encoded into three dependent layers, where two layers are sent on is encoded into three dependent layers, where two layers are sent on
the same Media Transport using different RTP Streams, i.e. SSRCs, and the same media transport using different RTP streams, i.e., SSRCs,
the third layer is sent on a separate Media Transport. and the third layer is sent on a separate media transport.
+----------------+ +----------------+
| Media Source | | Media Source |
+----------------+ +----------------+
| |
| |
V V
+---------------------------------------------------------+ +---------------------------------------------------------+
| Media Encoder | | Media Encoder |
+---------------------------------------------------------+ +---------------------------------------------------------+
skipping to change at page 28, line 48 skipping to change at page 30, line 48
+------+ +------+ | +------+ +------+ |
| | | | | |
V V V V V V
+-----------------+ +-----------------+ +-----------------+ +-----------------+
| Media Transport | | Media Transport | | Media Transport | | Media Transport |
+-----------------+ +-----------------+ +-----------------+ +-----------------+
Figure 9: Example of Media Source Layered Dependency Figure 9: Example of Media Source Layered Dependency
It is sometimes useful to make a distinction between using a single It is sometimes useful to make a distinction between using a single
Media Transport or multiple separate Media Transports when (in both media transport or multiple separate media transports when (in both
cases) using multiple RTP Streams to carry Encoded Streams and cases) using multiple RTP streams to carry encoded streams and
Dependent Streams for a Media Source. Therefore, the following new dependent streams for a media source. Therefore, the following new
terminology is defined here: terminology is defined here:
SRST: Single RTP Stream on a Single Media Transport SRST: Single RTP stream on a Single media Transport
MRST: Multiple RTP Streams on a Single Media Transport MRST: Multiple RTP streams on a Single media Transport
MRMT: Multiple RTP Streams on Multiple Media Transports MRMT: Multiple RTP streams on Multiple media Transports
MRST and MRMT relations needs to identify the common Media Encoder MRST and MRMT relations need to identify the common media encoder
origin for the Encoded and Dependent Streams. When using different origin for the encoded and dependent streams. When using different
RTP Sessions (MRMT), a single RTP Stream per Media Encoder, and a RTP sessions (MRMT), a single RTP stream per media encoder, and a
single Media Source in each RTP Session, common SSRC and CNAMEs can single media source in each RTP session, common SSRCs and CNAMEs can
be used to identify the common Media Source. When multiple RTP be used to identify the common media source. When multiple RTP
Streams are sent from one Media Encoder in the same RTP Session streams are sent from one media encoder in the same RTP session
(MRST), then CNAME is the only currently specified RTP identifier (MRST), then CNAME is the only currently specified RTP identifier
that can be used. In cases where multiple Media Encoders use that can be used. In cases where multiple media encoders use
multiple Media Sources sharing Synchronization Context, and thus multiple media sources sharing synchronization context, and thus have
having a common CNAME, additional heuristics or identification need a common CNAME, additional heuristics or identification need to be
to be applied to create the MRST or MRMT relationships between the applied to create the MRST or MRMT relationships between the RTP
RTP Streams. streams.
3.8. RTP Stream Duplication 3.8. RTP Stream Duplication
RTP Stream Duplication [RFC7198], using the same or different Media RTP Stream Duplication [RFC7198], using the same or different media
Transports, and optionally also delaying the duplicate [RFC7197], transports, and optionally also delaying the duplicate [RFC7197],
offers a simple way to protect media flows from packet loss in some offers a simple way to protect media flows from packet loss in some
cases (see Figure 10). This is a specific type of redundancy. All cases (see Figure 10). This is a specific type of redundancy. All
but one Source RTP Stream (Section 2.1.10) are effectively Redundancy but one source RTP stream (Section 2.1.10) are effectively redundancy
RTP Streams (Section 2.1.12), but since both Source and Redundant RTP RTP streams (Section 2.1.12), but since both source and redundant RTP
Streams are the same, it does not matter which one is which. This streams are the same, it does not matter which one is which. This
can also be seen as a specific type of Simulcast (Section 3.6) that can also be seen as a specific type of simulcast (Section 3.6) that
transmits the same Encoded Stream (Section 2.1.7) multiple times. transmits the same encoded stream (Section 2.1.7) multiple times.
+----------------+ +----------------+
| Media Source | | Media Source |
+----------------+ +----------------+
Source Stream | Source Stream |
V V
+----------------+ +----------------+
| Media Encoder | | Media Encoder |
+----------------+ +----------------+
Encoded Stream | Encoded Stream |
skipping to change at page 30, line 37 skipping to change at page 33, line 7
| |
V V
+-------------------+ +-------------------+
| Media Transport | | Media Transport |
+-------------------+ +-------------------+
Figure 10: Example of RTP Stream Duplication Figure 10: Example of RTP Stream Duplication
3.9. Redundancy Format 3.9. Redundancy Format
The RTP Payload for Redundant Audio Data [RFC2198] defines a "RTP Payload for Redundant Audio Data" [RFC2198] defines a transport
transport for redundant audio data together with primary data in the for redundant audio data together with primary data in the same RTP
same RTP payload. The redundant data can be a time delayed version payload. The redundant data can be a time-delayed version of the
of the primary or another time delayed Encoded Stream using a primary or another time-delayed encoded stream using a different
different Media Encoder to encode the same Media Source as the media encoder to encode the same media source as the primary, as
primary, as depicted in Figure 11. depicted in Figure 11.
+--------------------+ +--------------------+
| Media Source | | Media Source |
+--------------------+ +--------------------+
| |
Source Stream Source Stream
| |
+------------------------+ +------------------------+
| | | |
V V V V
skipping to change at page 31, line 31 skipping to change at page 33, line 40
| | | |
| +------------------+ | +------------------+
V V V V
+--------------------+ +--------------------+
| Media Packetizer | | Media Packetizer |
+--------------------+ +--------------------+
| |
V V
RTP Stream RTP Stream
Figure 11: Concept for usage of Audio Redundancy with different Media Figure 11: Concept for Usage of Audio Redundancy with Different Media
Encoders Encoders
The Redundancy format is thus providing the necessary meta The redundancy format is thus providing the necessary meta
information to correctly relate different parts of the same Encoded information to correctly relate different parts of the same encoded
Stream. The case depicted above (Figure 11) relates the Received stream. The case depicted above (Figure 11) relates the received
Source Stream fragments coming out of different Media Decoders, to be source stream fragments coming out of different media decoders, to be
able to combine them together into a less erroneous Source Stream. able to combine them together into a less erroneous source stream.
3.10. RTP Retransmission 3.10. RTP Retransmission
Figure 12 shows an example where a Media Source's Source RTP Stream Figure 12 shows an example where a media source's source RTP stream
is protected by a retransmission (RTX) flow [RFC4588]. In this is protected by a retransmission (RTX) flow [RFC4588]. In this
example the Source RTP Stream and the Redundancy RTP Stream share the example, the source RTP stream and the redundancy RTP stream share
same Media Transport. the same media transport.
+--------------------+ +--------------------+
| Media Source | | Media Source |
+--------------------+ +--------------------+
| |
V V
+--------------------+ +--------------------+
| Media Encoder | | Media Encoder |
+--------------------+ +--------------------+
| Retransmission | Retransmission
skipping to change at page 32, line 32 skipping to change at page 34, line 34
| | | |
+---------+ +---------+ +---------+ +---------+
| | | |
V V V V
+-----------------+ +-----------------+
| Media Transport | | Media Transport |
+-----------------+ +-----------------+
Figure 12: Example of Media Source Retransmission Flows Figure 12: Example of Media Source Retransmission Flows
The RTP Retransmission example (Figure 12) illustrates that this The RTP retransmission example (Figure 12) illustrates that this
mechanism works purely on the Source RTP Stream. The RTP mechanism works purely on the source RTP stream. The RTP
Retransmission transform buffers the sent Source RTP Stream and, upon retransmission transforms buffers from the sent source RTP stream
request, emits a retransmitted packet with an extra payload header as and, upon request, emits a retransmitted packet with an extra payload
a Redundancy RTP Stream. The RTP Retransmission mechanism [RFC4588] header as a redundancy RTP stream. The RTP retransmission mechanism
is specified such that there is a one to one relation between the [RFC4588] is specified such that there is a one-to-one relation
Source RTP Stream and the Redundancy RTP Stream. Therefore, a between the source RTP stream and the redundancy RTP stream.
Redundancy RTP Stream needs to be associated with its Source RTP Therefore, a redundancy RTP stream needs to be associated with its
Stream. This is done based on CNAME selectors and heuristics to source RTP stream. This is done based on CNAME selectors and
match requested packets for a given Source RTP Stream with the heuristics to match requested packets for a given source RTP stream
original sequence number in the payload of any new Redundancy RTP with the original sequence number in the payload of any new
Stream using the RTX payload format. In cases where the Redundancy redundancy RTP stream using the RTX payload format. In cases where
RTP Stream is sent in a different RTP Session than the Source RTP the redundancy RTP stream is sent in a different RTP session than the
Stream, the RTP Session relation is signaled by using the SDP Media source RTP stream, the RTP session relation is signaled by using the
Grouping's [RFC5888] Flow Identification (FID identification-tag) SDP media grouping's [RFC5888] Flow Identification (FID
semantics. identification-tag) semantics.
3.11. Forward Error Correction 3.11. Forward Error Correction
Figure 13 shows an example where two Media Sources' Source RTP Figure 13 shows an example where two media sources' source RTP
Streams are protected by Forward Error Correction (FEC). Source RTP streams are protected by FEC. Source RTP stream A has an RTP-based
Stream A has a RTP-based Redundancy transformation in FEC Encoder 1. redundancy transformation in FEC encoder 1. This produces a
This produces a Redundancy RTP Stream 1, that is only related to redundancy RTP stream 1, that is only related to source RTP stream A.
Source RTP Stream A. The FEC Encoder 2, however, takes two Source The FEC encoder 2, however, takes two source RTP streams (A and B)
RTP Streams (A and B) and produces a Redundancy RTP Stream 2 that and produces a redundancy RTP stream 2 that protects them jointly,
protects them jointly, i.e. Redundancy RTP Stream 2 relates to two i.e., redundancy RTP stream 2 relates to two source RTP streams (a
Source RTP Streams (a FEC group). FEC decoding, when needed due to FEC group). FEC decoding, when needed due to packet loss or packet
packet loss or packet corruption at the receiver, requires knowledge corruption at the receiver, requires knowledge about which source RTP
about which Source RTP Streams that the FEC encoding was based on. streams that the FEC encoding was based on.
In Figure 13 all RTP Streams are sent on the same Media Transport. In Figure 13, all RTP streams are sent on the same media transport.
This is however not the only possible choice. Numerous combinations This is, however, not the only possible choice. Numerous
exist for spreading these RTP Streams over different Media Transports combinations exist for spreading these RTP streams over different
to achieve the communication application's goal. media transports to achieve the communication application's goal.
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| Media Source A | | Media Source B | | Media Source A | | Media Source B |
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| | | |
V V V V
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| Media Encoder A | | Media Encoder B | | Media Encoder A | | Media Encoder B |
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| | | |
skipping to change at page 34, line 5 skipping to change at page 36, line 5
| +---------------+ +---------------+ | | +---------------+ +---------------+ |
| Redundancy | Redundancy | | | Redundancy | Redundancy | |
| RTP Stream 1 | RTP Stream 2 | | | RTP Stream 1 | RTP Stream 2 | |
V V V V V V V V
+----------------------------------------------------------+ +----------------------------------------------------------+
| Media Transport | | Media Transport |
+----------------------------------------------------------+ +----------------------------------------------------------+
Figure 13: Example of FEC Redundancy RTP Streams Figure 13: Example of FEC Redundancy RTP Streams
As FEC Encoding exists in various forms, the methods for relating FEC As FEC encoding exists in various forms, the methods for relating FEC
Redundancy RTP Streams with its source information in Source RTP redundancy RTP streams with its source information in source RTP
Streams are many. The XOR based RTP FEC Payload format [RFC5109] is streams are many. The XOR-based RTP FEC payload format [RFC5109] is
defined in such a way that a Redundancy RTP Stream has a one to one defined in such a way that a redundancy RTP stream has a one-to-one
relation with a Source RTP Stream. In fact, the RFC requires the relation with a source RTP stream. In fact, the RFC requires the
Redundancy RTP Stream to use the same SSRC as the Source RTP Stream. redundancy RTP stream to use the same SSRC as the source RTP stream.
This requires the use of either a separate RTP Session, or the This requires the use of either a separate RTP session or the
Redundancy RTP Payload format [RFC2198]. The underlying relation redundancy RTP payload format [RFC2198]. The underlying relation
requirement for this FEC format and a particular Redundancy RTP requirement for this FEC format and a particular redundancy RTP
Stream is to know the related Source RTP Stream, including its SSRC. stream is to know the related source RTP stream, including its SSRC.
3.12. RTP Stream Separation 3.12. RTP Stream Separation
RTP Streams can be separated exclusively based on their SSRCs, at the RTP streams can be separated exclusively based on their SSRCs, at the
RTP Session level, or at the Multi-Media Session level. RTP session level, or at the multimedia session level.
When the RTP Streams that have a relationship are all sent in the When the RTP streams that have a relationship are all sent in the
same RTP Session and are uniquely identified based on their SSRC same RTP session and are uniquely identified based on their SSRC
only, it is termed an SSRC-Only Based Separation. Such streams can only, it is termed an "SSRC-only-based separation". Such streams can
be related via RTCP CNAME to identify that the streams belong to the be related via RTCP CNAME to identify that the streams belong to the
same Endpoint. SSRC-based approaches [RFC5576], when used, can same endpoint. SSRC-based approaches [RFC5576], when used, can
explicitly relate various such RTP Streams. explicitly relate various such RTP streams.
On the other hand, when RTP Streams that are related are sent in the On the other hand, when RTP streams that are related are sent in the
context of different RTP Sessions to achieve separation, it is known context of different RTP sessions to achieve separation, it is known
as RTP Session-based separation. This is commonly used when the as "RTP session-based separation". This is commonly used when the
different RTP Streams are intended for different Media Transports. different RTP streams are intended for different media transports.
Several mechanisms that use RTP Session-based separation rely on it Several mechanisms that use RTP session-based separation rely on it
to enable an implicit grouping mechanism expressing the relationship. as a grouping mechanism expressing the relationship. The solutions
The solutions have been based on using the same SSRC value in the have been based on using the same SSRC value in the different RTP
different RTP Sessions to implicitly indicate their relation. That sessions to implicitly indicate their relation. That way, no
way, no explicit RTP level mechanism has been needed, only signaling explicit RTP level mechanism has been needed; only signaling level
level relations have been established using semantics from Grouping relations have been established using semantics from the media-line
of Media lines framework [RFC5888]. Examples of this are RTP grouping framework [RFC5888]. Examples of this are RTP
Retransmission [RFC4588], SVC Multi-Session Transmission [RFC6190] retransmission [RFC4588], SVC Multi-Session Transmission [RFC6190],
and XOR Based FEC [RFC5109]. RTCP CNAME explicitly relates RTP and XOR-based FEC [RFC5109]. RTCP CNAME explicitly relates RTP
Streams across different RTP Sessions, as explained in the previous streams across different RTP sessions, as explained in the previous
section. Such a relationship can be used to perform inter-media section. Such a relationship can be used to perform inter-media
synchronization. synchronization.
RTP Streams that are related and need to be associated can be part of RTP streams that are related and need to be associated can be part of
different Multimedia Sessions, rather than just different RTP different multimedia sessions, rather than just different RTP
Sessions within the same Multimedia Session context. This puts sessions within the same multimedia session context. This puts
further demand on the scope of the mechanism(s) and its handling of further demand on the scope of the mechanism(s) and its handling of
identifiers used for expressing the relationships. identifiers used for expressing the relationships.
3.13. Multiple RTP Sessions over one Media Transport 3.13. Multiple RTP Sessions over one Media Transport
[I-D.westerlund-avtcore-transport-multiplexing] describes a mechanism [TRANSPORT-MULTIPLEX] describes a mechanism that allows several RTP
that allows several RTP Sessions to be carried over a single sessions to be carried over a single underlying media transport. The
underlying Media Transport. The main reasons for doing this are main reasons for doing this are related to the impact of using one or
related to the impact of using one or more Media Transports (using a more media transports (using a common network path or potentially
common network path or potentially have different ones). The fewer having different ones). The fewer media transports used, the less
Media Transports used, the less need for NAT/FW traversal resources need for NAT/firewall traversal resources and smaller number of flow-
and smaller number of flow based Quality of Service (QoS). based QoS.
However, Multiple RTP Sessions over one Media Transport imply that a However, multiple RTP sessions over one media transport imply that a
single Media Transport 5-tuple is not sufficient to express in which single media transport 5-tuple is not sufficient to express in which
RTP Session context a particular RTP Stream exists. Complexities in RTP session context a particular RTP stream exists. Complexities in
the relationship between Media Transports and RTP Session already the relationship between media transports and RTP sessions already
exist as one RTP Session contains multiple Media Transports, e.g. exist as one RTP session contains multiple media transports, e.g.,
even a Peer-to-Peer RTP Session with RTP/RTCP Multiplexing requires even a Peer-to-Peer RTP Session with RTP/RTCP Multiplexing requires
two Media Transports, one in each direction. The relationship two media transports, one in each direction. The relationship
between Media Transports and RTP Sessions as well as additional between media transports and RTP sessions as well as additional
levels of identifiers need to be considered in both signaling design levels of identifiers needs to be considered in both signaling design
and when defining terminology. and when defining terminology.
4. Mapping from Existing Terms 4. Mapping from Existing Terms
This section describes a selected set of terms from some relevant This section describes a selected set of terms from some relevant
IETF RFC and Internet Drafts (at the time of writing), using the RFCs and Internet-Drafts (at the time of writing), using the concepts
concepts from previous sections. from previous sections.
4.1. Telepresence Terms 4.1. Telepresence Terms
The terms in this sub-section are used in the context of CLUE The terms in this subsection are used in the context of CLUE
[I-D.ietf-clue-framework]. Note that some terms listed in this sub- [CLUE-FRAME]. Note that some terms listed in this subsection use the
section use the same names as terms defined elsewhere in this same names as terms defined elsewhere in this document. Unless
document. Unless explicitly stated (as "RTP Taxonomy") and in this explicitly stated (as "RTP Taxonomy") and in this subsection, they
sub-section, they are to be read as references to the CLUE-specific are to be read as references to the CLUE-specific term within this
term within this sub-section. subsection.
4.1.1. Audio Capture 4.1.1. Audio Capture
Defined in CLUE as a Media Capture (Section 4.1.7) for audio. Defined in CLUE as a Media Capture (Section 4.1.7) for audio.
Describes an audio Media Source (Section 2.1.4). Describes an audio media source (Section 2.1.4).
4.1.2. Capture Device 4.1.2. Capture Device
Defined in CLUE as a device that converts physical input into an Defined in CLUE as a device that converts physical input into an
electrical signal. Identifies a physical entity performing an RTP electrical signal. Identifies a physical entity performing an RTP
Taxonomy Media Capture (Section 2.1.2) transformation. Taxonomy media capture (Section 2.1.2) transformation.
4.1.3. Capture Encoding 4.1.3. Capture Encoding
Defined in CLUE as a specific encoding (Section 4.1.6) of a Media Defined in CLUE as a specific Encoding (Section 4.1.6) of a Media
Capture (Section 4.1.7). Describes an Encoded Stream (Section 2.1.7) Capture (Section 4.1.7). Describes an encoded stream (Section 2.1.7)
related to CLUE specific semantic information. related to CLUE-specific semantic information.
4.1.4. Capture Scene 4.1.4. Capture Scene
Defined in CLUE as a structure representing a spatial region captured Defined in CLUE as a structure representing a spatial region captured
by one or more Capture Devices (Section 4.1.2), each capturing media by one or more Capture Devices (Section 4.1.2), each capturing media
representing a portion of the region. Describes a set of spatially representing a portion of the region. Describes a set of spatially
related Media Sources (Section 2.1.4). related media sources (Section 2.1.4).
4.1.5. Endpoint 4.1.5. Endpoint
Defined in CLUE as a CLUE-capable device which is the logical point Defined in CLUE as a CLUE-capable device that is the logical point of
of final termination through receiving, decoding and rendering and/or final termination through receiving, decoding, and rendering and/or
initiation through capturing, encoding, and sending of media streams initiation through capturing, encoding, and sending of media Streams
(Section 4.1.10). CLUE further defines it to consist of one or more (Section 4.1.10). CLUE further defines it to consist of one or more
physical devices with source and sink media streams, and exactly one physical devices with source and sink media streams, and exactly one
[RFC4353] Participant. Describes exactly one Participant participant [RFC4353]. Describes exactly one participant
(Section 2.2.3) and one or more RTP Taxonomy Endpoints (Section 2.2.3) and one or more RTP Taxonomy endpoints
(Section 2.2.1). (Section 2.2.1).
4.1.6. Individual Encoding 4.1.6. Individual Encoding
Defined in CLUE as a set of parameters representing a way to encode a Defined in CLUE as a set of parameters representing a way to encode a
Media Capture (Section 4.1.7) to become a Capture Encoding Media Capture (Section 4.1.7) to become a Capture Encoding
(Section 4.1.3). Describes the configuration information needed to (Section 4.1.3). Describes the configuration information needed to
perform a Media Encoder (Section 2.1.6) transformation. perform a media encoder (Section 2.1.6) transformation.
4.1.7. Media Capture 4.1.7. Media Capture
Defined in CLUE as a source of media, such as from one or more Defined in CLUE as a source of media, such as from one or more
Capture Devices (Section 4.1.2) or constructed from other media Capture Devices (Section 4.1.2) or constructed from other media
streams (Section 4.1.10). Describes either an RTP Taxonomy Media Streams (Section 4.1.10). Describes either an RTP Taxonomy media
Capture (Section 2.1.2) or a Media Source (Section 2.1.4), depending capture (Section 2.1.2) or a media source (Section 2.1.4), depending
on in which context the term is used. on in which context the term is used.
4.1.8. Media Consumer 4.1.8. Media Consumer
Defined in CLUE as a CLUE-capable device that intends to receive Defined in CLUE as a CLUE-capable device that intends to receive
Capture Encodings (Section 4.1.3). Describes the media receiving Capture Encodings (Section 4.1.3). Describes the media receiving
part of an RTP Taxonomy Endpoint (Section 2.2.1). part of an RTP Taxonomy endpoint (Section 2.2.1).
4.1.9. Media Provider 4.1.9. Media Provider
Defined in CLUE as a CLUE-capable device that intends to send Capture Defined in CLUE as a CLUE-capable device that intends to send Capture
Encodings (Section 4.1.3). Describes the media sending part of an Encodings (Section 4.1.3). Describes the media sending part of an
RTP Taxonomy Endpoint (Section 2.2.1). RTP Taxonomy endpoint (Section 2.2.1).
4.1.10. Stream 4.1.10. Stream
Defined in CLUE as a Capture Encoding (Section 4.1.3) sent from a Defined in CLUE as a Capture Encoding (Section 4.1.3) sent from a
Media Provider (Section 4.1.9) to a Media Consumer (Section 4.1.8) Media Provider (Section 4.1.9) to a Media Consumer (Section 4.1.8)
via RTP. Describes an RTP Stream (Section 2.1.10). via RTP. Describes an RTP stream (Section 2.1.10).
4.1.11. Video Capture 4.1.11. Video Capture
Defined in CLUE as a Media Capture (Section 4.1.7) for video. Defined in CLUE as a Media Capture (Section 4.1.7) for video.
Describes a video Media Source (Section 2.1.4). Describes a video media source (Section 2.1.4).
4.2. Media Description 4.2. Media Description
A single Session Description Protocol (SDP) [RFC4566] media A single Session Description Protocol (SDP) [RFC4566] Media
description (or media block; an m-line and all subsequent lines until Description (or media block; an "m=" line and all subsequent lines
the next m-line or the end of the SDP) describes part of the until the next "m=" line or the end of the SDP) describes part of the
necessary configuration and identification information needed for a necessary configuration and identification information needed for a
Media Encoder transformation, as well as the necessary configuration media encoder transformation, as well as the necessary configuration
and identification information for the Media Decoder to be able to and identification information for the media decoder to be able to
correctly interpret a received RTP Stream. correctly interpret a received RTP stream.
A Media Description typically relates to a single Media Source. This A media description typically relates to a single media source. This
is for example an explicit restriction in WebRTC. However, nothing is, for example, an explicit restriction in WebRTC. However, nothing
prevents that the same Media Description (and same RTP Session) is prevents that the same media description (and same RTP session) is
re-used for multiple Media Sources reused for multiple media sources [RTP-MULTI-STREAM]. It can thus
[I-D.ietf-avtcore-rtp-multi-stream]. It can thus describe properties describe properties of one or more RTP streams, and can also describe
of one or more RTP Streams, and can also describe properties valid properties valid for an entire RTP session (via [RFC5576] mechanisms,
for an entire RTP Session (via [RFC5576] mechanisms, for example). for example).
4.3. Media Stream 4.3. Media Stream
RTP [RFC3550] uses media stream, audio stream, video stream, and RTP [RFC3550] uses media stream, audio stream, video stream, and a
stream of (RTP) packets interchangeably, which are all RTP Streams. stream of (RTP) packets interchangeably, which are all RTP streams.
4.4. Multimedia Conference 4.4. Multimedia Conference
A Multimedia Conference is a Communication Session (Section 2.2.5) A Multimedia Conference is a communication session (Section 2.2.5)
between two or more Participants (Section 2.2.3), along with the between two or more participants (Section 2.2.3), along with the
software they are using to communicate. software they are using to communicate.
4.5. Multimedia Session 4.5. Multimedia Session
SDP [RFC4566] defines a Multimedia Session as a set of multimedia SDP [RFC4566] defines a multimedia session as a set of multimedia
senders and receivers and the data streams flowing from senders to senders and receivers and the data streams flowing from senders to
receivers, which would correspond to a set of Endpoints and the RTP receivers, which would correspond to a set of endpoints and the RTP
Streams that flow between them. In this document, Multimedia Session streams that flow between them. In this document, multimedia session
(Section 2.2.4) also assumes those Endpoints belong to a set of (Section 2.2.4) also assumes those endpoints belong to a set of
Participants that are engaged in communication via a set of related participants that are engaged in communication via a set of related
RTP Streams. RTP streams.
RTP [RFC3550] defines a Multimedia Session as a set of concurrent RTP RTP [RFC3550] defines a multimedia session as a set of concurrent RTP
Sessions among a common group of Participants. For example, a video sessions among a common group of participants. For example, a video
conference may contain an audio RTP Session and a video RTP Session. conference may contain an audio RTP session and a video RTP session.
This would correspond to a group of Participants (each using one or This would correspond to a group of participants (each using one or
more Endpoints) sharing a set of concurrent RTP Sessions. In this more endpoints) sharing a set of concurrent RTP sessions. In this
document, Multimedia Session also defines those RTP Sessions to have document, multimedia session also defines those RTP sessions to have
some relation and be part of a communication among the Participants. some relation and be part of a communication among the participants.
4.6. Multipoint Control Unit (MCU) 4.6. Multipoint Control Unit (MCU)
This term is commonly used to describe the central node in any type This term is commonly used to describe the central node in any type
of star topology [I-D.ietf-avtcore-rtp-topologies-update] conference. of star topology [RTP-TOPOLOGIES] conference. It describes a device
It describes a device that includes one Participant (Section 2.2.3) that includes one participant (Section 2.2.3) (usually corresponding
(usually corresponding to a so-called conference focus) and one or to a so-called conference focus) and one or more related endpoints
more related Endpoints (Section 2.2.1) (sometimes one or more per (Section 2.2.1) (sometimes one or more per conference participant).
conference Participant).
4.7. Multi-Session Transmission (MST) 4.7. Multi-Session Transmission (MST)
One of two transmission modes defined in H.264 based SVC [RFC6190], One of two transmission modes defined in H.264-based SVC [RFC6190],
the other mode being SST (Section 4.13). In Multi-Session the other mode being a Single-Session Transmission (SST)
Transmission (MST), the SVC Media Encoder sends Encoded Streams and (Section 4.14). In Multi-Session Transmission (MST), the SVC media
Dependent Streams distributed across two or more RTP Streams in one encoder sends encoded streams and dependent streams distributed
or more RTP Sessions. The term "MST" is ambiguous in RFC 6190, across two or more RTP streams in one or more RTP sessions. The term
especially since the name indicates the use of multiple "sessions", "MST" is ambiguous in RFC 6190, especially since the name indicates
while MST type packetization is in fact required whenever two or more the use of multiple "sessions", while MST-type packetization is in
RTP Streams are used for the Encoded and Dependent Streams, fact required whenever two or more RTP streams are used for the
regardless if those are sent in one or more RTP Sessions. encoded and dependent streams, regardless if those are sent in one or
Corresponds either to MRST or MRMT (Section 3.7) stream relations more RTP sessions. Corresponds either to MRST or MRMT (Section 3.7)
defined in this document. The SVC RTP Payload RFC [RFC6190] is not stream relations defined in this document. The SVC RTP payload RFC
particularly explicit about how the common Media Encoder [RFC6190] is not particularly explicit about how the common media
(Section 2.1.6) relation between Encoded Streams (Section 2.1.7) and encoder (Section 2.1.6) relation between encoded streams
Dependent Streams (Section 2.1.8) is to be implemented. (Section 2.1.7) and dependent streams (Section 2.1.8) is to be
implemented.
4.8. Recording Device 4.8. Recording Device
WebRTC specifications use this term to refer to locally available WebRTC specifications use this term to refer to locally available
entities performing a Media Capture (Section 2.1.2) transformation. entities performing a media capture (Section 2.1.2) transformation.
4.9. RtcMediaStream 4.9. RtcMediaStream
A WebRTC RtcMediaStream is a set of Media Sources (Section 2.1.4) A WebRTC RtcMediaStream is a set of media sources (Section 2.1.4)
sharing the same Synchronization Context (Section 3.1). sharing the same synchronization context (Section 3.1).
4.10. RtcMediaStreamTrack 4.10. RtcMediaStreamTrack
A WebRTC RtcMediaStreamTrack is a Media Source (Section 2.1.4). A WebRTC RtcMediaStreamTrack is a media source (Section 2.1.4).
4.11. RTP Sender 4.11. RTP Receiver
RTP [RFC3550] uses this term, which can be seen as the RTP protocol RTP [RFC3550] uses this term, which can be seen as the RTP protocol
part of a Media Packetizer (Section 2.1.9). part of a media depacketizer (Section 2.1.27).
4.12. RTP Session 4.12. RTP Sender
Within the context of SDP, a singe m= line can map to a single RTP RTP [RFC3550] uses this term, which can be seen as the RTP protocol
Session (Section 2.2.2) or multiple m= lines can map to a single RTP part of a media packetizer (Section 2.1.9).
Session. The latter is enabled via multiplexing schemes such as
BUNDLE [I-D.ietf-mmusic-sdp-bundle-negotiation], for example, which
allows mapping of multiple m= lines to a single RTP Session.
4.13. Single Session Transmission (SST) 4.13. RTP Session
One of two transmission modes defined in H.264 based SVC [RFC6190], Within the context of SDP, a singe "m=" line can map to a single RTP
the other mode being MST (Section 4.7). In Single Session session (Section 2.2.2), or multiple "m=" lines can map to a single
Transmission (SST), the SVC Media Encoder sends Encoded Streams RTP session. The latter is enabled via multiplexing schemes such as
(Section 2.1.7) and Dependent Streams (Section 2.1.8) combined into a BUNDLE [SDP-BUNDLE], for example, which allows mapping of multiple
single RTP Stream (Section 2.1.10) in a single RTP Session "m=" lines to a single RTP session.
(Section 2.2.2), using the SVC RTP Payload format. The term "SST" is
ambiguous in RFC 6190, in that it sometimes refers to the use of a
single RTP Stream, like in sections relating to packetization, and
sometimes appears to refer to use of a single RTP Session, like in
the context of discussing SDP. Closely corresponds to SRST
(Section 3.7) defined in this document.
4.14. SSRC 4.14. Single-Session Transmission (SST)
One of two transmission modes defined in H.264-based SVC [RFC6190],
the other mode being MST (Section 4.7). In SST, the SVC media
encoder sends encoded streams (Section 2.1.7) and dependent streams
(Section 2.1.8) combined into a single RTP stream (Section 2.1.10) in
a single RTP session (Section 2.2.2), using the SVC RTP payload
format. The term "SST" is ambiguous in RFC 6190, in that it
sometimes refers to the use of a single RTP stream, like in sections
relating to packetization, and sometimes appears to refer to use of a
single RTP session, like in the context of discussing SDP. Closely
corresponds to SRST (Section 3.7) defined in this document.
4.15. SSRC
RTP [RFC3550] defines this as "the source of a stream of RTP RTP [RFC3550] defines this as "the source of a stream of RTP
packets", which indicates that an SSRC is not only a unique packets", which indicates that an SSRC is not only a unique
identifier for the Encoded Stream (Section 2.1.7) carried in those identifier for the encoded stream (Section 2.1.7) carried in those
packets, but is also effectively used as a term to denote a Media packets but is also effectively used as a term to denote a media
Packetizer (Section 2.1.9). In [RFC3550], it is stated that "a packetizer (Section 2.1.9). In [RFC3550], it is stated that "a
synchronization source may change its data format, e.g., audio synchronization source may change its data format, e.g., audio
encoding, over time". The related Encoded Stream data format in an encoding, over time". The related encoded stream data format in an
RTP Stream (Section 2.1.10) is identified by the RTP Payload Type. RTP stream (Section 2.1.10) is identified by the RTP payload type.
Changing data format for an Encoded Stream effectively also changes Changing the data format for an encoded stream effectively also
what Media Encoder (Section 2.1.6) that is used for the Encoded changes what media encoder (Section 2.1.6) is used for the encoded
Stream. No ambiguity is introduced to SSRC as Encoded Stream stream. No ambiguity is introduced to SSRC as an encoded stream
identifier by allowing RTP Payload Type changes, as long as only a identifier by allowing RTP payload type changes, as long as only a
single RTP Payload Type is valid for any given RTP Time Stamp. This single RTP payload type is valid for any given RTP Timestamp. This
is aligned with and further described by Section 5.2 of [RFC3550]. is aligned with and further described by Section 5.2 of [RFC3550].
5. Security Considerations 5. Security Considerations
The purpose of this document is to make clarifications and reduce the The purpose of this document is to make clarifications and reduce the
confusion prevalent in RTP taxonomy because of inconsistent usage by confusion prevalent in RTP taxonomy because of inconsistent usage by
multiple technologies and protocols making use of the RTP protocol. multiple technologies and protocols making use of the RTP protocol.
It does not introduce any new security considerations beyond those It does not introduce any new security considerations beyond those
already well documented in the RTP protocol [RFC3550] and each of the already well documented in the RTP protocol [RFC3550] and each of the
many respective specifications of the various protocols making use of many respective specifications of the various protocols making use of
it. it.
Having a well-defined common terminology and understanding of the Having a well-defined common terminology and understanding of the
complexities of the RTP architecture will help lead us to better complexities of the RTP architecture will help lead us to better
standards, avoiding security problems. standards, avoiding security problems.
6. Acknowledgement 6. Informative References
This document has many concepts borrowed from several documents such
as WebRTC [I-D.ietf-rtcweb-overview], CLUE [I-D.ietf-clue-framework],
and Multiplexing Architecture
[I-D.westerlund-avtcore-transport-multiplexing]. The authors would
like to thank all the authors of each of those documents.
The authors would also like to acknowledge the insights, guidance and
contributions of Magnus Westerlund, Roni Even, Paul Kyzivat, Colin
Perkins, Keith Drage, Harald Alvestrand, Alex Eleftheriadis, Mo
Zanaty, Stephan Wenger, and Bernard Aboba.
7. Contributors
Magnus Westerlund has contributed the concept model for the media
chain using transformations and streams model, including rewriting
pre-existing concepts into this model and adding missing concepts.
The first proposal for updating the relationships and the topologies
based on this concept was also performed by Magnus.
8. IANA Considerations
This document makes no request of IANA.
9. Informative References
[I-D.ietf-avtcore-rtp-multi-stream]
Lennox, J., Westerlund, M., Wu, W., and C. Perkins,
"Sending Multiple Media Streams in a Single RTP Session",
draft-ietf-avtcore-rtp-multi-stream-08 (work in progress),
July 2015.
[I-D.ietf-avtcore-rtp-topologies-update]
Westerlund, M. and S. Wenger, "RTP Topologies", draft-
ietf-avtcore-rtp-topologies-update-10 (work in progress),
July 2015.
[I-D.ietf-clue-framework] [CLUE-FRAME]
Duckworth, M., Pepperell, A., and S. Wenger, "Framework Duckworth, M., Pepperell, A., and S. Wenger, "Framework
for Telepresence Multi-Streams", draft-ietf-clue- for Telepresence Multi-Streams", Work in Progress,
framework-22 (work in progress), April 2015. draft-ietf-clue-framework-22, April 2015.
[I-D.ietf-mmusic-sdp-bundle-negotiation]
Holmberg, C., Alvestrand, H., and C. Jennings,
"Negotiating Media Multiplexing Using the Session
Description Protocol (SDP)", draft-ietf-mmusic-sdp-bundle-
negotiation-23 (work in progress), July 2015.
[I-D.ietf-mmusic-sdp-simulcast]
Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty,
"Using Simulcast in SDP and RTP Sessions", draft-ietf-
mmusic-sdp-simulcast-00 (work in progress), January 2015.
[I-D.ietf-rtcweb-overview]
Alvestrand, H., "Overview: Real Time Protocols for
Browser-based Applications", draft-ietf-rtcweb-overview-14
(work in progress), June 2015.
[I-D.westerlund-avtcore-transport-multiplexing]
Westerlund, M. and C. Perkins, "Multiplexing Multiple RTP
Sessions onto a Single Lower-Layer Transport", draft-
westerlund-avtcore-transport-multiplexing-07 (work in
progress), October 2013.
[RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., [RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V.,
Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse- Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse-
Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, Parisis, "RTP Payload for Redundant Audio Data", RFC 2198,
DOI 10.17487/RFC2198, September 1997, DOI 10.17487/RFC2198, September 1997,
<http://www.rfc-editor.org/info/rfc2198>. <http://www.rfc-editor.org/info/rfc2198>.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
skipping to change at page 44, line 5 skipping to change at page 44, line 43
[RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP [RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP
Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014, Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
<http://www.rfc-editor.org/info/rfc7201>. <http://www.rfc-editor.org/info/rfc7201>.
[RFC7273] Williams, A., Gross, K., van Brandenburg, R., and H. [RFC7273] Williams, A., Gross, K., van Brandenburg, R., and H.
Stokking, "RTP Clock Source Signalling", RFC 7273, Stokking, "RTP Clock Source Signalling", RFC 7273,
DOI 10.17487/RFC7273, June 2014, DOI 10.17487/RFC7273, June 2014,
<http://www.rfc-editor.org/info/rfc7273>. <http://www.rfc-editor.org/info/rfc7273>.
Appendix A. Changes From Earlier Versions [RTP-MULTI-STREAM]
Lennox, J., Westerlund, M., Wu, W., and C. Perkins,
NOTE TO RFC EDITOR: Please remove this section prior to publication. "Sending Multiple Media Streams in a Single RTP Session",
Work in Progress, draft-ietf-avtcore-rtp-multi-stream-08,
A.1. Modifications Between WG Version -07 and -08 July 2015.
Addresses comments from IESG evaluation.
o Made text more firm around what improvements this document
introduces.
o Clarified the distinction between analog and digital in sections
2.1.1 and 2.1.2.
o Removed the explicit requirement that a Source RTP Stream must
send at least some data from an Encoded Stream, replacing it with
a statement that it is directly related to the Encoded Stream.
o Moved the clarification that RTP-based Redundancy excludes Media
Encoder redundancy data in an Encoded Stream from Section 2.1.10
(RTP Stream) to 2.1.11 (RTP-based Redundancy), since that
statement applies to RTP-based Redundancy rather than to RTP
Stream.
o Added clarification that a Media Transport Sender can
intentionally pace packet transmission.
o Aligned text around delay variation to use this term throughout,
and added a reference to RFC 5481.
o Added that RTP Session is a group communications channel that can
potentially carry a number of RTP Streams, as an additional
clarification below Figure 7.
o Added a clarification in Section 4.1 around Telepresence Terms on
which references are to CLUE terms and which are to other sections
of this document, for terms that have the same name in CLUE as in
this document.
o Clarified in Section 4.14 what SSRC data format changes means,
since the RFC 3550 SSRC definition mentions this possibility.
o Editorial improvements.
A.2. Modifications Between WG Version -06 and -07
Addresses comments from AD review and GenArt review.
o Added RTP-based Security and RTP-based Validation transform
sections, as well as Secured RTP Stream and Received Secured RTP
Stream sections.
o Improved wording in Abstract and Introduction sections.
o Clarified what is considered "media" in section 2.1.2 Media
Capture.
o Changed a number of "Characteristics" lists to more suitable prose
text.
o Re-worded text around use of Encoded and Dependent RTP Streams in
section 2.1.9 Media Packetizer.
o Clarified description of Source RTP Stream in section 2.1.10.
o Clarified motivation to use separate Media Transports for
Simulcast in section 3.6.
o Added local descriptions of terms imported from CLUE framework.
o Editorial improvements.
A.3. Modifications Between WG Version -05 and -06
o Clarified that a Redundancy RTP Stream can be used standalone to
generate Repaired RTP Streams.
o Clarified that (in accordance with above) RTP-based Repair takes
zero or more Received RTP Streams and one or more Received
Redundancy RTP Streams as input.
o Changed Figure 6 to more clearly show that Media Transport is
terminated in the Endpoint, not in the Participant.
o Added a sentence to Endpoint section that clarifies there may be
contexts where a single "host" can serve multiple Participants,
making those Endpoints share some properties.
o Merged previous section 3.5 on SST/MST with previous section 3.8
on Layered Multi-Stream into a common section discussing the
scalable/layered stream relation, and moved improved, descriptive
text on SST and MST to new sub-sections 4.7 and 4.13, describing
them as existing terms.
o Editorial improvements.
A.4. Modifications Between WG Version -04 and -05
o Editorial improvements.
A.5. Modifications Between WG Version -03 and -04
o Changed "Media Redundancy" and "Media Repair" to "RTP-based
Redundancy" and "RTP-based Repair", since those terms are more
specific and correct.
o Changed "End Point" to "Endpoint" and removed Editor's Note on
this.
o Clarified that a Media Capture may impose constraints on clock
handling.
o Clarified that mixing multiple Raw Streams into a Source Stream is
not possible, since that requires mixed streams to have a timing
relation, requiring them to be Source Streams, and added an
example.
o Clarified that RTP-based Redundancy excludes the type of encoding
redundancy found within the encoded media format in an Encoded
Stream.
o Clarified that a Media Transport contains only a single RTP
Session, but a single RTP Session can span multiple Media
Transports.
o Clarified that packets with seemingly correct checksum that are
received by a Media Transport Receiver may still be corrupt.
o Clarified that a corrupt packet in a Media Transport Receiver is
typically either discarded or somehow marked and passed on in the
Received RTP Stream.
o Added Synchronization Context to Figure 6.
o Editorial improvements and clarifications.
A.6. Modifications Between WG Version -02 and -03
o Changed section 3.5, removing SST-SS/MS and MST-SS/MS, replacing
them with SRST, MRST, and MRMT.
o Updated section 3.8 to align with terminology changes in section
3.5.
o Added a new section 4.12, describing the term Multimedia
Conference.
o Changed reference from I-D to now published RFC 7273.
o Editorial improvements and clarifications.
A.7. Modifications Between WG Version -01 and -02
o Major re-structure
o Moved media chain Media Transport detailing up one section level
o Collapsed level 2 sub-sections of section 3 and thus moved level 3
sub-sections up one level, gathering some introductory text into
the beginning of section 3
o Added that not only SSRC collision, but also a clock rate change
[RFC7160] is a valid reason to change SSRC value for an RTP stream
o Added a sub-section on clock source signaling
o Added a sub-section on RTP stream duplication
o Elaborated a bit in section 2.2.1 on the relation between End
Points, Participants and CNAMEs
o Elaborated a bit in section 2.2.4 on Multimedia Session and
synchronization contexts
o Removed the section on CLUE scenes defining an implicit
synchronization context, since it was incorrect
o Clarified text on SVC SST and MST according to list discussions
o Removed the entire topology section to avoid possible
inconsistencies or duplications with draft-ietf-avtcore-rtp-
topologies-update, but saved one example overview figure of
Communication Entities into that section
o Added a section 4 on mapping from existing terms with one sub-
section per term, mainly by moving text from sections 2 and 3
o Changed all occurrences of Packet Stream to RTP Stream
o Moved all normative references to informative, since this is an
informative document
o Added references to RFC 7160, RFC 7197 and RFC 7198, and removed
unused references
A.8. Modifications Between WG Version -00 and -01
o WG version -00 text is identical to individual draft -03
o Amended description of SVC SST and MST encodings with respect to
concepts defined in this text
o Removed UML as normative reference, since the text no longer uses
any UML notation
o Removed a number of level 4 sections and moved out text to the
level above
A.9. Modifications Between Version -02 and -03
o Section 4 rewritten (and new communication topologies added) to [RTP-TOPOLOGIES]
reflect the major updates to Sections 1-3 Westerlund, M. and S. Wenger, "RTP Topologies", Work in
Progress, draft-ietf-avtcore-rtp-topologies-update-10,
July 2015.
o Section 8 removed (carryover from initial -00 draft) [SDP-BUNDLE]
Holmberg, C., Alvestrand, H., and C. Jennings,
"Negotiating Media Multiplexing Using the Session
Description Protocol (SDP)", Work in Progress,
draft-ietf-mmusic-sdp-bundle-negotiation-23, July 2015.
o General clean up of text, grammar and nits [SDP-SIMULCAST]
Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty,
"Using Simulcast in SDP and RTP Sessions", Work in
Progress, draft-ietf-mmusic-sdp-simulcast-01, July 2015.
A.10. Modifications Between Version -01 and -02 [TRANSPORT-MULTIPLEX]
Westerlund, M. and C. Perkins, "Multiplexing Multiple RTP
Sessions onto a Single Lower-Layer Transport", Work in
Progress, draft-westerlund-avtcore-transport-multiplexing-
07, October 2013.
o Section 2 rewritten to add both streams and transformations in the [WEBRTC-OVERVIEW]
media chain. Alvestrand, H., "Overview: Real Time Protocols for
Browser-based Applications", Work in Progress,
draft-ietf-rtcweb-overview-14, June 2015.
o Section 3 rewritten to focus on exposing relationships. Acknowledgements
A.11. Modifications Between Version -00 and -01 This document has many concepts borrowed from several documents such
as WebRTC [WEBRTC-OVERVIEW], CLUE [CLUE-FRAME], and Multiplexing
Architecture [TRANSPORT-MULTIPLEX]. The authors would like to thank
all the authors of each of those documents.
o Too many to list The authors would also like to acknowledge the insights, guidance,
and contributions of Magnus Westerlund, Roni Even, Paul Kyzivat,
Colin Perkins, Keith Drage, Harald Alvestrand, Alex Eleftheriadis, Mo
Zanaty, Stephan Wenger, and Bernard Aboba.
o Added new authors Contributors
o Updated content organization and presentation Magnus Westerlund has contributed the concept model for the media
chain using transformations and streams model, including rewriting
pre-existing concepts into this model and adding missing concepts.
The first proposal for updating the relationships and the topologies
based on this concept was also performed by Magnus.
Authors' Addresses Authors' Addresses
Jonathan Lennox Jonathan Lennox
Vidyo, Inc. Vidyo, Inc.
433 Hackensack Avenue 433 Hackensack Avenue
Seventh Floor Seventh Floor
Hackensack, NJ 07601 Hackensack, NJ 07601
US United States
Email: jonathan@vidyo.com Email: jonathan@vidyo.com
Kevin Gross Kevin Gross
AVA Networks, LLC AVA Networks, LLC
Boulder, CO Boulder, CO
US United States
Email: kevin.gross@avanw.com Email: kevin.gross@avanw.com
Suhas Nandakumar Suhas Nandakumar
Cisco Systems Cisco Systems
170 West Tasman Drive 170 West Tasman Drive
San Jose, CA 95134 San Jose, CA 95134
US United States
Email: snandaku@cisco.com Email: snandaku@cisco.com
Gonzalo Salgueiro Gonzalo Salgueiro
Cisco Systems Cisco Systems
7200-12 Kit Creek Road 7200-12 Kit Creek Road
Research Triangle Park, NC 27709 Research Triangle Park, NC 27709
US United States
Email: gsalguei@cisco.com Email: gsalguei@cisco.com
Bo Burman (editor) Bo Burman (editor)
Ericsson Ericsson
Kistavagen 25 Kistavagen 25
SE-16480 Stockholm SE-16480 Stockholm
Sweden Sweden
Email: bo.burman@ericsson.com Email: bo.burman@ericsson.com
 End of changes. 261 change blocks. 
1087 lines changed or deleted 840 lines changed or added

This html diff was produced by rfcdiff 1.42. The latest version is available from http://tools.ietf.org/tools/rfcdiff/