draft-ietf-payload-rtp-jpegxs-02.txt   draft-ietf-payload-rtp-jpegxs-03.txt 
Payload Working Group S. Lugan avtcore S. Lugan
Internet-Draft G. Rouvroy Internet-Draft A. Descampe
Intended status: Standards Track A. Descampe Intended status: Standards Track C. Damman
Expires: April 11, 2020 intoPIX Expires: October 10, 2020 intoPIX
T. Richter T. Richter
IIS IIS
A. Willeme A. Willeme
UCL/ICTEAM UCL/ICTEAM
October 9, 2019 April 8, 2020
RTP Payload Format for ISO/IEC 21122 (JPEG XS) RTP Payload Format for ISO/IEC 21122 (JPEG XS)
draft-ietf-payload-rtp-jpegxs-02 draft-ietf-payload-rtp-jpegxs-03
Abstract Abstract
This document specifies a Real-Time Transport Protocol (RTP) payload This document specifies a Real-Time Transport Protocol (RTP) payload
format to be used for transporting JPEG XS (ISO/IEC 21122) encoded format to be used for transporting JPEG XS (ISO/IEC 21122) encoded
video. JPEG XS is a low-latency, lightweight image coding system. video. JPEG XS is a low-latency, lightweight image coding system.
Compared to an uncompressed video use case, it allows higher Compared to an uncompressed video use case, it allows higher
resolutions and frame rates, while offering visually lossless resolutions and frame rates, while offering visually lossless
quality, reduced power consumption, and end-to-end latency confined quality, reduced power consumption, and end-to-end latency confined
to a fraction of a frame. to a fraction of a frame.
skipping to change at page 1, line 41 skipping to change at page 1, line 41
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 11, 2020. This Internet-Draft will expire on October 10, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions, Definitions, and Abbreviations . . . . . . . . . 3 2. Conventions, Definitions, and Abbreviations . . . . . . . . . 3
3. Media Format Description . . . . . . . . . . . . . . . . . . 4 3. Media Format Description . . . . . . . . . . . . . . . . . . 4
3.1. Image Data Structures . . . . . . . . . . . . . . . . . . 4 3.1. Image Data Structures . . . . . . . . . . . . . . . . . . 4
3.2. Codestream . . . . . . . . . . . . . . . . . . . . . . . 5 3.2. Codestream . . . . . . . . . . . . . . . . . . . . . . . 5
3.3. Video support box and colour specification box . . . . . 5 3.3. Video support box and colour specification box . . . . . 5
4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 5 3.4. JPEG XS Frame . . . . . . . . . . . . . . . . . . . . . . 5
4.1. Payload Header . . . . . . . . . . . . . . . . . . . . . 6 4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 6
4.2. Payload Data . . . . . . . . . . . . . . . . . . . . . . 8 4.1. RTP packetization . . . . . . . . . . . . . . . . . . . . 6
4.3. Traffic Shaping and Delivery Timing . . . . . . . . . . . 10 4.2. Payload Header . . . . . . . . . . . . . . . . . . . . . 8
5. Congestion Control Considerations . . . . . . . . . . . . . . 10 4.3. Payload Data . . . . . . . . . . . . . . . . . . . . . . 11
6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 10 4.4. Traffic Shaping and Delivery Timing . . . . . . . . . . . 14
6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 10 5. Congestion Control Considerations . . . . . . . . . . . . . . 14
6.2. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . 13 6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 14
6.2.1. General . . . . . . . . . . . . . . . . . . . . . . . 13 6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 14
6.2.2. Media type and subtype . . . . . . . . . . . . . . . 14 6.2. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . 17
6.2.3. Traffic shaping . . . . . . . . . . . . . . . . . . . 14 6.2.1. General . . . . . . . . . . . . . . . . . . . . . . . 17
6.2.4. Offer/Answer Considerations . . . . . . . . . . . . . 14 6.2.2. Media type and subtype . . . . . . . . . . . . . . . 18
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15 6.2.3. Traffic shaping . . . . . . . . . . . . . . . . . . . 18
8. Security Considerations . . . . . . . . . . . . . . . . . . . 15 6.2.4. Offer/Answer Considerations . . . . . . . . . . . . . 18
9. RFC Editor Considerations . . . . . . . . . . . . . . . . . . 16 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 8. Security Considerations . . . . . . . . . . . . . . . . . . . 19
10.1. Normative References . . . . . . . . . . . . . . . . . . 16 9. RFC Editor Considerations . . . . . . . . . . . . . . . . . . 20
10.2. Informative References . . . . . . . . . . . . . . . . . 18 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 20
10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 18 10.1. Normative References . . . . . . . . . . . . . . . . . . 20
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 18 10.2. Informative References . . . . . . . . . . . . . . . . . 22
10.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22
1. Introduction 1. Introduction
This document specifies a payload format for packetization of JPEG XS This document specifies a payload format for packetization of JPEG XS
encoded video signals into the Real-time Transport Protocol (RTP) encoded video signals into the Real-time Transport Protocol (RTP)
[RFC3550]. [RFC3550].
The JPEG XS coding system offers compression and recompression of The JPEG XS coding system offers compression and recompression of
image sequences with very moderate computational resources while image sequences with very moderate computational resources while
remaining robust under multiple compression and decompression cycles remaining robust under multiple compression and decompression cycles
skipping to change at page 3, line 20 skipping to change at page 3, line 22
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Application Data Unit (ADU) Application Data Unit (ADU)
The unit of source data provided as payload to the transport The unit of source data provided as payload to the transport
layer, and corresponding, in this RTP payload definition, to a layer, and corresponding, in this RTP payload definition, to a
single JPEG XS frame. single JPEG XS frame.
Colour specification box Colour specification box (CS box)
A ISO colour specification box defined in ISO/IEC 21122-3 A ISO colour specification box defined in ISO/IEC 21122-3
[ISO21122-3] that includes colour-related metadata required to [ISO21122-3] that includes colour-related metadata required to
correctly display JPEG XS frames, such as colour primaries, correctly display JPEG XS frames, such as colour primaries,
transfer characteristics and matrix coefficients. transfer characteristics and matrix coefficients.
EOC marker
A marker that consists of the two bytes 0xff11 indicating the end
of a JPEG XS codestream.
JPEG XS codestream JPEG XS codestream
A sequence of bytes representing a compressed image formatted A sequence of bytes representing a compressed image formatted
according to JPEG XS Part 1 [ISO21122-1], except the End-Of- according to JPEG XS Part-1 [ISO21122-1].
Codestream (EOC) marker which is omitted in this payload format.
JPEG XS codestream header JPEG XS codestream header
A sequence of bytes at the beginning of each JPEG XS codestream A sequence of bytes, starting with a SOC marker, at the beginning
encoded in multiple markers and marker segments that does not of each JPEG XS codestream encoded in multiple markers and marker
carry entropy coded data, but metadata such as the frame dimension segments that does not carry entropy coded data, but metadata such
and component precision. as the frame dimension and component precision.
JPEG XS frame JPEG XS frame
The concatenation of a video support box, as defined in JPEG XS The concatenation of a video support box, as defined in ISO/IEC
Part 3 [ISO21122-3], a colour specification box, as defined as 21122-3 [ISO21122-3], a colour specification box, as defined in
well in JPEG XS Part 3 [ISO21122-3] and a JPEG XS codestream. ISO/IEC 21122-3 [ISO21122-3] as well, and either one JPEG XS
codestream in the case of a progressive frame or two JPEG XS
codestreams in the case of an interlaced frame.
JPEG XS header segment JPEG XS header segment
The concatenation of a video support box, as defined in JPEG XS The concatenation of a video support box, as defined in ISO/IEC
Part 3 [ISO21122-3], a colour specification box, as defined as 21122-3 [ISO21122-3], a colour specification box, as defined in
well in JPEG XS Part 3 [ISO21122-3] and a JPEG XS codestream ISO/IEC 21122-3 as well [ISO21122-3] and a JPEG XS codestream
header. header.
JPEG XS stream JPEG XS stream
A sequence of JPEG XS frames A sequence of JPEG XS frames.
Marker Marker
A two-byte functional sequence that is part of a JPEG XS A two-byte functional sequence that is part of a JPEG XS
codestream starting with a 0xff byte and a subsequent byte codestream starting with a 0xff byte and a subsequent byte
defining its function. defining its function.
Marker segment Marker segment
A marker along with a 16-bit marker size and payload data A marker along with a 16-bit marker size and payload data
following the size. following the size.
Packetization unit
A portion of a Application Data Unit whose boundaries shall
coincide with boundaries of RTP packet payloads, i.e. the first
(resp. last) byte of a packetization unit shall be the first
(resp. last) byte of a RTP packet payload.
Slice Slice
The smallest independently decodable unit of a JPEG XS codestream, The smallest independently decodable unit of a JPEG XS codestream,
bearing in mind that it decodes to wavelet coefficients which bearing in mind that it decodes to wavelet coefficients which
still require inverse wavelet filtering to give an image. still require inverse wavelet filtering to give an image.
SOC marker SOC marker
A marker that consists of the two bytes 0xff10 indicating the A marker that consists of the two bytes 0xff10 indicating the
start of a JPEG XS codestream. start of a JPEG XS codestream.
Video support box Video support box (VS box)
A ISO video support box defined in ISO/IEC 21122-3 [ISO21122-3] A ISO video support box defined in ISO/IEC 21122-3 [ISO21122-3]
that includes metadata required to play back a JPEG XS stream, that includes metadata required to play back a JPEG XS stream,
such as its maximum bitrate, its subsampling structure, its buffer such as its maximum bitrate, its subsampling structure, its buffer
model and its frame rate. model and its frame rate.
3. Media Format Description 3. Media Format Description
3.1. Image Data Structures 3.1. Image Data Structures
JPEG XS is a low-latency lightweight image coding system for coding JPEG XS is a low-latency lightweight image coding system for coding
skipping to change at page 4, line 47 skipping to change at page 5, line 12
where each band consists of multiple coefficients describing the where each band consists of multiple coefficients describing the
image signal of a given component within a frequency domain specific image signal of a given component within a frequency domain specific
to the wavelet filter type, i.e. the particular filter corresponding to the wavelet filter type, i.e. the particular filter corresponding
to the band. to the band.
Wavelet coefficients are grouped into precincts, where each precinct Wavelet coefficients are grouped into precincts, where each precinct
includes all coefficients over all bands that contribute to a spatial includes all coefficients over all bands that contribute to a spatial
region of the image. region of the image.
One or multiple precincts are furthermore combined into slices One or multiple precincts are furthermore combined into slices
consisting of an integral number of precincts. Precincts do not consisting of an integer number of precincts. Precincts do not cross
cross slice boundaries, and wavelet coefficients in precincts that slice boundaries, and wavelet coefficients in precincts that are part
are part of different slices can be decoded independently from each of different slices can be decoded independently from each other.
other. Note, however, that the wavelet transformation runs across Note, however, that the wavelet transformation runs across slice
slice boundaries. A slice always extends over the full width of the boundaries. A slice always extends over the full width of the image,
image, but may only cover parts of its height. but may only cover parts of its height.
Each JPEG XS frame consists of a JPEG XS header segment followed by
one or multiple slices completely describing a single frame.
3.2. Codestream 3.2. Codestream
A JPEG XS codestream header, followed by several slices, and
terminated by an EOC marker form a JPEG XS codestream.
The overall codestream format, including the definition of all The overall codestream format, including the definition of all
markers, is further defined in ISO/IEC 21122-1 [ISO21122-1]. It markers, is further defined in ISO/IEC 21122-1 [ISO21122-1]. It
represents sample values of a single frame, bare any interpretation represents sample values of a single image, bare any interpretation
relative to a colour space. relative to a colour space.
3.3. Video support box and colour specification box 3.3. Video support box and colour specification box
While the information defined in the codestream is sufficient to While the information defined in the codestream is sufficient to
reconstruct the sample values of one video frame, the interpretation reconstruct the sample values of one image, the interpretation of the
of the samples remains undefined by the codestream itself. This samples remains undefined by the codestream itself. This
interpretation is given by the video support box and the colour interpretation is given by the video support box and the colour
specification box which contain significant information to correctly specification box which contain significant information to correctly
play the JPEG XS stream. The layout and syntax of these boxes, play the JPEG XS stream. The layout and syntax of these boxes,
together with their content, are defined in ISO/IEC 21122-3 together with their content, are defined in ISO/IEC 21122-3
[ISO21122-3]. The video support box provides information on the [ISO21122-3]. The video support box provides information on the
maximum bitrate, the frame rate, the subsampling image format, the maximum bitrate, the frame rate, the subsampling image format, the
timecode of the current JPEG XS frame, the profile, level and timecode of the current JPEG XS frame, the profile, level and
sublevel used (as defined in ISO/IEC 21122-2 [ISO21122-2]), and sublevel used (as defined in ISO/IEC 21122-2 [ISO21122-2]), and
optionally on the buffer model and the mastering display metadata. optionally on the buffer model and the mastering display metadata.
The colour specification box indicates the colour primaries, transfer The colour specification box indicates the colour primaries, transfer
characteristics, matrix coefficients and video full range flag needed characteristics, matrix coefficients and video full range flag needed
to specify the colour space of the video stream. to specify the colour space of the video stream.
3.4. JPEG XS Frame
The concatenation of a video support box, a colour specification box
and one or two JPEG XS codestreams forms a JPEG XS frame. In the
case of a video stream made of progressive frames, only one
codestream follows the boxes. In the case of a video stream made of
interlaced frames, two codestreams follow the boxes, each
corresponding to a field of the interlaced frame. The video
information box included in the video support box contains a frat
field indicating if the frame is progressive or interlaced (see ISO/
IEC 21122-3 [ISO21122-3]). This information can also be found in
each RTP packet header (see Section 4.2).
4. Payload Format 4. Payload Format
This section specifies the payload format for JPEG XS streams over This section specifies the payload format for JPEG XS streams over
the Real-time Transport Protocol (RTP) [RFC3550]. the Real-time Transport Protocol (RTP) [RFC3550].
In order to be transported over RTP, each JPEG XS stream is In order to be transported over RTP, each JPEG XS stream is
transported in a distinct RTP stream, identified by a distinct SSRC. transported in a distinct RTP stream, identified by a distinct SSRC.
A JPEG XS stream is divided into Application Data Units (ADUs), each A JPEG XS stream is divided into Application Data Units (ADUs), each
ADU corresponding to a single JPEG XS frame. ADU corresponding to a single JPEG XS frame.
An ADU is split into multiple RTP packet payloads. Figure 1 shows an 4.1. RTP packetization
example of how a JPEG XS frame fits into the payload of RTP packets
("Hdr" denotes a RTP packet header). As seen there, each packet An ADU is made of several packetization units. If a packetization
contains either part of the JPEG XS header segment or part of a unit is bigger than the maximum size of a RTP packet payload, the
single slice. Both may extend over multiple packets. The payload of unit is split into multiple RTP packet payloads, as illustrated in
every packet shall have the same size (based e.g. on the Maximum Figure 1. As seen there, each packet shall contain (part of) one and
Transfer Unit of the network), except (possibly) the last packet of only one packetization unit. A packetization unit may extend over
the JPEG XS header segment or a slice. The boundaries of the JPEG XS multiple packets. The payload of every packet shall have the same
header segment and of every slice shall coincide with the boundaries size (based e.g. on the Maximum Transfer Unit of the network), except
of the payload of a packet, i.e. the first (resp. last) byte of the (possibly) the last packet of a packetization unit. The boundaries
JPEG XS header segment or a slice shall be the first (resp. last) of a packetization unit shall coincide with the boundaries of the
byte of the payload. payload of a packet, i.e. the first (resp. last) byte of the
packetization unit shall be the first (resp. last) byte of the
payload.
RTP +-----+------------------------+ RTP +-----+------------------------+
Packet #1 | Hdr | JPEG XS header segment | Packet #1 | Hdr | Packetization unit #1 |
+-----+------------------------+ +-----+------------------------+
RTP +-----+---------------------------+ RTP +-----+--------------------------------------+
Packet #2 | Hdr | Slice 0 | Packet #2 | Hdr | Packetization unit #2 |
+-----+---------------------------+ +-----+--------------------------------------+
RTP +-----+---------------------------------------------+ RTP +-----+--------------------------------------------------+
Packet #3 | Hdr | Slice 1 (part 1/3) | Packet #3 | Hdr | Packetization unit #3 (part 1/3) |
+-----+---------------------------------------------+ +-----+--------------------------------------------------+
RTP +-----+---------------------------------------------+ RTP +-----+--------------------------------------------------+
Packet #4 | Hdr | Slice 1 (part 2/3) | Packet #4 | Hdr | Packetization unit #3 (part 2/3) |
+-----+---------------------------------------------+ +-----+--------------------------------------------------+
RTP +-----+---------------------+ RTP +-----+----------------------------------------------+
Packet #5 | Hdr | Slice 1 (part 3/3) | Packet #5 | Hdr | Packetization unit #3 (part 3/3) |
+-----+---------------------+ +-----+----------------------------------------------+
... ...
RTP +-----+-----------------------+ RTP +-----+-----------------------------------------+
Packet #P | Hdr | Slice N-1 (part q/q) | Packet #P | Hdr | Packetization unit #N (part q/q) |
+-----+-----------------------+ +-----+-----------------------------------------+
Figure 1: Example of ADU defining a single JPEG XS frame Figure 1: Example of ADU packetization
4.1. Payload Header There are two different packetization modes defined for this RTP
payload format.
Figure 2 illustrates the RTP payload header used in order to 1. Codestream packetization mode: in this mode, the packetization
unit shall be the entire codestream, preceeded by boxes, if any.
This means that a progressive frame will have a single
packetization unit, while an interlaced frame will have two. The
progressive case is illustrated in Figure 2.
2. Slice packetization mode: in this mode, the packetization unit
shall be the slice, i.e. there shall be data from no more than
one slice per RTP packet. The first packetization unit shall be
made of the JPEG XS header segment (i.e. the concatenation of the
VS box, the CS box and the JPEG XS codestream header). This
first unit is then followed by successive units, each containing
one and only one slice. The packetization unit containing the
last slice of a JPEG XS codestream shall also contain the EOC
marker immediately following this last slice. This is
illustrated in Figure 3. In the case of interlaced frame, the
JPEG XS codestream header of the second field shall be in its own
packetization unit.
RTP +-----+--------------------------------------------------+
Packet #1 | Hdr | VS box + CS box + JPEG XS codestream (part 1/q) |
+-----+--------------------------------------------------+
RTP +-----+--------------------------------------------------+
Packet #2 | Hdr | JPEG XS codestream (part 2/q) |
+-----+--------------------------------------------------+
...
RTP +-----+--------------------------------------+
Packet #P | Hdr | JPEG XS codestream (part q/q) |
+-----+--------------------------------------+
Figure 2: Example of codestream packetization mode
RTP +-----+----------------------------+
Packet #1 | Hdr | JPEG XS header segment |
+-----+----------------------------+
RTP +-----+--------------------------------------------------+
Packet #2 | Hdr | Slice #1 (part 1/2) |
+-----+--------------------------------------------------+
RTP +-----+-------------------------------------------+
Packet #3 | Hdr | Slice #1 (part 2/2) |
+-----+-------------------------------------------+
RTP +-----+--------------------------------------------------+
Packet #4 | Hdr | Slice #2 (part 1/3) |
+-----+--------------------------------------------------+
...
RTP +-----+---------------------------------------+
Packet #P | Hdr | Slice #N (part q/q) + EOC marker |
+-----+---------------------------------------+
Figure 3: Example of slice packetization mode
Thanks to the constant bit-rate of JPEG XS, the codestream
packetization mode guarantees that a JPEG XS RTP stream will produce
a constant number of bytes per frame, and a constant number of RTP
packets per frame. To reach the same guarantee with the slice
packetization mode, an additional constraint needs to be set at the
rate allocation stage in the JPEG XS encoder. For instance, one
option would be to impose a constant bit-rate at the slice level.
4.2. Payload Header
Figure 4 illustrates the RTP payload header used in order to
transport a JPEG XS stream. transport a JPEG XS stream.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+---+-+-+-------+-+-------------+-------------------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| V |P|X| CC |M| PT | Sequence number | | V |P|X| CC |M| PT | sequence number |
+---+-+-+-------+-+-------------+-------------------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp | | timestamp |
+---------------------------------------------------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Synchronization source (SSRC) identifier | | synchronization source (SSRC) identifier |
+===============================================================+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| Contributing source (CSRC) identifiers | | contributing source (CSRC) identifiers |
| .... | | .... |
+-+-------------+-----------------------+-----------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|L|Frame counter| Slice counter | Packet counter | |T|K|L| I |F counter| SEP counter | P counter |
+-+-------------+-----------------------+-----------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Data |
+---------------------------------------------------------------+
Figure 2: RTP and payload headers Figure 4: RTP and payload headers
The version (V), padding (P), extension (X), CSRC count (CC), The version (V), padding (P), extension (X), CSRC count (CC),
sequence number, synchronization source (SSRC) and contributing sequence number, synchronization source (SSRC) and contributing
source (CSRC) fields follow their respective definitions in RFC 3550 source (CSRC) fields follow their respective definitions in RFC 3550
[RFC3550]. [RFC3550].
The timestamp SHOULD be based on a 90 kHz clock reference. The timestamp SHOULD be based on a 90 kHz clock reference.
As per specified in RFC 3550 [RFC3550] and RFC 4175 [RFC4175], the As per specified in RFC 3550 [RFC3550] and RFC 4175 [RFC4175], the
RTP timestamp designates the sampling instant of the first octet of RTP timestamp designates the sampling instant of the first octet of
skipping to change at page 7, line 49 skipping to change at page 9, line 47
frame begins. frame begins.
If the sampling instant does not correspond to an integer value of If the sampling instant does not correspond to an integer value of
the clock, the value shall be truncated to the next lowest integer, the clock, the value shall be truncated to the next lowest integer,
with no ambiguity. with no ambiguity.
The remaining fields are defined as follows: The remaining fields are defined as follows:
Marker (M) [1 bit]: Marker (M) [1 bit]:
The M bit is used to indicate the last packet of a frame. This The M bit is used to indicate the last packet of a frame. This
enables a decoder to finish decoding the frame, where it otherwise enables a decoder to finish decoding the frame.
may need to wait for the next packet to explicitly know that the
frame is finished.
Payload Type (PT) [7 bits]: Payload Type (PT) [7 bits]:
A dynamically allocated payload type field that designates the A dynamically allocated payload type field that designates the
payload as JPEG XS video. payload as JPEG XS video.
Transmission mode (T) [1 bit]:
The T bit is set to indicate that packets are sent sequentially by
the transmitter. A receiver could use this information to
dimension its input buffer(s) accordingly. If T=0, nothing can be
assumed about the transmission order and packets may be sent out-
of-order by the transmitter. If T=1, packets must be sent
sequentially by the transmitter.
pacKetization mode (K) [1 bit]:
The K bit is set to indicate which packetization mode is used.
K=0 indicates codestream packetization mode, while K=1 indicates
slice packetization mode. If Transmission mode (T) is set to 0,
slice packetization mode must be used and K must be set to 1.
Last (L) [1 bit]: Last (L) [1 bit]:
The L bit is set to indicate the last packet of the JPEG XS header The L bit is set to indicate the last packet of a packetization
segment or a slice. It enables the decoder to already start unit. As the end of the frame also ends the packet containing the
decoding a slice without having to wait for the full frame to last unit of the frame, the L bit is set whenever the M bit is
finish, and thus allows low-latency decoding. As the end of the set. In the case of a progressive frame using the codestream
frame also ends the packet containing the last slice of the frame, packetization mode, the L bit and M bit are equivalent.
the L bit is set whenever the M bit is set.
Frame counter [7 bits]: Interlaced mode (I) [2 bit]:
This field identifies the frame number modulo 128 to which a These 2 bits are used to indicate how the JPEG XS frame is scanned
packet belongs. Frame numbers increment by 1 for each frame (progressive or interlaced).
transmitted. The frame number, in addition to the time stamp, may
help the decoder to manage its input buffer and to bring packets
back into their natural order.
Slice counter [12 bits]: 00: The payload is progressively scanned.
This field identifies the slice modulo 4096 to which the packet
contributes. If the data belongs to the JPEG XS header segment,
this field shall have its maximal value, namely 4095 = 0x0fff.
Otherwise, it is the slice index modulo 4096. Slice indices count
from 0 at the top of the frame to their maximum number.
Packet counter [12 bits]: 01: Reserved for future use.
This field identifies the packet number modulo 4096 within the
JPEG XS header segment or a slice. The packet counter is set to 0
at the start of the JPEG XS header segment and incremented by 1
for every subsequent packet (if any) of this JPEG XS header
segment. The packet counter is then reset to 0 at the start of
every slice, and incremented by 1 for every packet that
contributes to the same slice.
4.2. Payload Data 10: The payload is part of the first field of an interlaced video
frame. The height specified in the JPEG XS picture header is
half of the height of the entire displayed image.
11: The payload is part of the second field of an interlaced
video frame. The height specified in the JPEG XS picture header
is half of the height of the entire displayed image.
F counter [5 bits]:
The frame (F) counter identifies the frame number modulo 32 to
which a packet belongs. Frame numbers are incremented by 1 for
each frame transmitted. The frame number, in addition to the time
stamp, may help the decoder manage its input buffer and bring
packets back into their natural order.
SEP counter [11 bits]:
The Slice and Extended Packet (SEP) counter is used differently
depending on the packetization mode.
* In the case of codestream packetization mode (K=0), this
counter resets whenever the Packet counter resets (see
hereunder), and increments by 1 whenever the Packet counter
overruns.
* In the case of slice packetization mode (K=1), this counter
identifies the slice modulo 2047 to which the packet
contributes. If the data belongs to the JPEG XS header
segment, this field shall have its maximal value, namely 2047 =
0x07ff. Otherwise, it is the slice index modulo 2047. Slice
indices are counted from 0 (corresponding to the top of the
frame).
P counter [11 bits]:
The packet (P) counter identifies the packet number modulo 2048
within the current packetization unit. It is set to 0 at the
start of the packetization unit and incremented by 1 for every
subsequent packet (if any) belonging to the same unit.
Practically, if codestream packetization mode is enabled, this
field counts the packets within a codestream and is extended by
the SEP counter when it overruns. If slice packetization mode is
enabled, this field counts the packets within a slice or within
the JPEG XS header segment.
4.3. Payload Data
The payload data of a JPEG XS RTP stream consists of a concatenation The payload data of a JPEG XS RTP stream consists of a concatenation
of multiple JPEG XS frames. of multiple JPEG XS frames.
Each JPEG XS frame is the concatenation of a JPEG XS header segment Each JPEG XS frame is the concatenation of one or more packetization
followed by one or several slices completely defining a single frame. unit(s), as explained in Section 4.1. Figure 5 depicts this layout
Figure 3 depicts this layout. for an interlaced frame in the codestream packetization mode and
Figure 6 depicts this layout for a progressive frame in the slice
packetization mode. The Frame counter value is not indicated because
the value is constant for all packetization units of a given frame.
+--------[ JPEG XS header segment ]---------+ +=====[ Packetization unit (PU) #1 ]====+
| Video support box | Slice counter = 0x0fff | Video support box | SEP counter = 0
| +-------------------------------------+ | | +---------------------------------+ | P counter = 0
| | Sub boxes of the video support box | | | : Sub boxes of the VS box : |
| +-------------------------------------+ | | +---------------------------------+ |
| : additional sub-boxes of the vs-box : | +- - - - - - - - - - - - - - - - - - - -+
| +-------------------------------------+ | | Colour specification box |
| | | +---------------------------------+ |
+-------------------------------------------+ | : Fields of the CS box : |
| Colour specification box | | +---------------------------------+ |
| +-------------------------------------+ | +- - - - - - - - - - - - - - - - - - - -+
| | Specification method (METH = 5) | | | JPEG XS codestream (field 0) |
| +-------------------------------------+ | : (part 1/q) : M=0, K=0, L=0, I=10
| : additional fields of the cs-box : | +---------------------------------------+
| +-------------------------------------+ | | JPEG XS codestream (field 0) | SEP counter = 0
| | | (part 2/q) | P counter = 1
+-------------------------------------------+ : : M=0, K=0, L=0, I=10
| JPEG XS codestream header | +---------------------------------------+
| +-------------------------------------+ | | JPEG XS codestream (field 0) | SEP counter = 0
| | SOC marker | | | (part 3/q) | P counter = 2
| +-------------------------------------+ | : : M=0, K=0, L=0, I=10
| : Additional Marker segments : | +---------------------------------------+
| +-------------------------------------+ | : :
| | M = 0, L = 1 +---------------------------------------+
+-------------------------------------------+ | JPEG XS codestream (field 0) | SEP counter = 1
+----------------[ Slices ]-----------------+ | (part 2049/q) | P counter = 0
| Slice #0 | Slice counter = 0 : : M=0, K=0, L=0, I=10
| +-------------------------------------+ | +---------------------------------------+
| | SLH Marker | | : :
| +-------------------------------------+ | +---------------------------------------+
| : Entropy Coded Data : | | JPEG XS codestream (field 0) | SEP counter = (q-1) div 2048
| | | | | (part q/q) | P counter = (q-1) mod 2048
| +-------------------------------------+ | : : M=0, K=0, L=1, I=10
| | M = 0, L = 1 +===============[ PU #2 ]===============+
+-------------------------------------------+ | JPEG XS codestream (field 1) | SEP counter = 0
| Slice #1 | Slice counter = 1 | (part 1/q) | P counter = 0
: : M = 0, L = 1 : : M=0, K=0, L=0, I=11
+-------------------------------------------+ +---------------------------------------+
: : | JPEG XS codestream (field 1) | SEP counter = 0
+-------------------------------------------+ | (part 2/q) | P counter = 1
| Slice #N-1 | Slice counter = N-1 : : M=0, K=0, L=0, I=11
: : M = 1, L = 1 +---------------------------------------+
+-------------------------------------------+ : :
+---------------------------------------+
| JPEG XS codestream (field 1) | SEP counter = (q-1) div 2048
| (part q/q) | P counter = (q-1) mod 2048
: : M=1, K=0, L=1, I=11
+=======================================+
Figure 3: JPEG XS Payload Data Figure 5: Example of JPEG XS Payload Data (codestream packetization
mode, interlaced frame)
4.3. Traffic Shaping and Delivery Timing +====[ PU#1: JPEG XS Header segment ]===+
| Video support box | SEP counter = 0x07FF
| +---------------------------------+ | P counter = 0
| : Sub boxes of the VS box : |
| +---------------------------------+ |
+- - - - - - - - - - - - - - - - - - - -+
| Colour specification box |
| +---------------------------------+ |
| : Fields of the CS box : |
| +---------------------------------+ |
+- - - - - - - - - - - - - - - - - - - -+
| JPEG XS codestream header |
| +---------------------------------+ |
| : Markers and marker segments : |
| +---------------------------------+ | M=0, T=0, K=1, L=1, I=00
+==========[ PU#2: Slice #1 ]===========+
| +---------------------------------+ | SEP counter = 0
| | SLH Marker | | P counter = 0
| +---------------------------------+ |
| : Entropy Coded Data : |
| +---------------------------------+ | M=0, T=0, K=1, L=1, I=00
+==========[ PU#3: Slice #2 ]===========+
| Slice #2 | SEP counter = 1
| (part 1/q) | P counter = 0
: : M=0, T=0, K=1, L=0, I=00
+---------------------------------------+
| Slice #2 | SEP counter = 1
| (part 2/q) | P counter = 1
: : M=0, T=0, K=1, L=0, I=00
+---------------------------------------+
: :
+---------------------------------------+
| Slice #2 | SEP counter = 1
| (part q/q) | P counter = q-1
: : M=0, T=0, K=1, L=1, I=00
+=======================================+
: :
+========[ PU#N: Slice #(N-1) ]=========+
| Slice #(N-1) | SEP counter = N-2
| (part 1/r) | P counter = 0
: : M=0, T=0, K=1, L=0, I=00
+---------------------------------------+
: :
+---------------------------------------+
| Slice #(N-1) | SEP counter = N-2
| (part r/r) | P counter = r-1
: + EOC marker : M=1, T=0, K=1, L=1, I=00
+=======================================+
Figure 6: Example of JPEG XS Payload Data (slice packetization mode,
progressive frame)
4.4. Traffic Shaping and Delivery Timing
The traffic shaping and delivery timing shall be in accordance with The traffic shaping and delivery timing shall be in accordance with
the Network Compatibility Model compliance definitions specified in the Network Compatibility Model compliance definitions specified in
SMPTE ST 2110-21 [SMPTE-ST2110-21] for either Narrow Linear Senders SMPTE ST 2110-21 [SMPTE-ST2110-21] for either Narrow Linear Senders
(Type NL) or Wide Senders (Type W). The session description shall (Type NL) or Wide Senders (Type W). The session description shall
include a format-specific parameter of either TP=2110TPNL or include a format-specific parameter of either TP=2110TPNL or
TP=2110TPW to indicate compliance with Type NL or Type W TP=2110TPW to indicate compliance with Type NL or Type W
respectively. respectively.
NOTE: The Virtual Receiver Buffer Model compliance definitions of ST NOTE: The Virtual Receiver Buffer Model compliance definitions of ST
skipping to change at page 10, line 44 skipping to change at page 14, line 44
Type name: video Type name: video
Subtype name: jxsv Subtype name: jxsv
Required parameters: Required parameters:
rate: The RTP timestamp clock rate. Applications using this rate: The RTP timestamp clock rate. Applications using this
payload format SHOULD use a value of 90000. payload format SHOULD use a value of 90000.
transmission mode: Indicates if packets are sent sequentially by
the transmitter. A receiver could use this information to
dimension its input buffer(s) accordingly. If set to 0, nothing
can be assumed about the transmission order and packets may be sent
out-of-order. If value is 1, packets must be sent sequentially by
the transmitter.
Optional parameters: Optional parameters:
profile: The JPEG XS profile in use, as defined in ISO/IEC 21122-2 profile: The JPEG XS profile in use, as defined in ISO/IEC 21122-2
(JPEG XS Part 2) [ISO21122-2]. (JPEG XS Part 2) [ISO21122-2].
level: The JPEG XS level in use, as defined in ISO/IEC 21122-2 level: The JPEG XS level in use, as defined in ISO/IEC 21122-2
(JPEG XS Part 2) [ISO21122-2]. (JPEG XS Part 2) [ISO21122-2].
sublevel: The JPEG XS sublevel in use, as defined in ISO/IEC sublevel: The JPEG XS sublevel in use, as defined in ISO/IEC
21122-2 (JPEG XS Part 2) [ISO21122-2]. 21122-2 (JPEG XS Part 2) [ISO21122-2].
skipping to change at page 14, line 12 skipping to change at page 18, line 16
specific mapping to fields in the Session Description Protocol (SDP), specific mapping to fields in the Session Description Protocol (SDP),
which is commonly used to describe RTP sessions. which is commonly used to describe RTP sessions.
6.2.2. Media type and subtype 6.2.2. Media type and subtype
The media type ("video") goes in SDP "m=" as the media name. The media type ("video") goes in SDP "m=" as the media name.
The media subtype ("jxsv") goes in SDP "a=rtpmap" as the encoding The media subtype ("jxsv") goes in SDP "a=rtpmap" as the encoding
name, followed by a slash ("/") and the required parameter "rate" name, followed by a slash ("/") and the required parameter "rate"
corresponding to the RTP timestamp clock rate (which for the payload corresponding to the RTP timestamp clock rate (which for the payload
format defined in this document MUST be 90000). The optional format defined in this document SHOULD be 90000), followed by a slash
parameters go in the SDP "a=fmtp" attribute by copying them directly ("/") and the required parameter "transmission mode" set to 1 if
from the MIME media type string as a semicolon-separated list of packets are sent sequentially by the transmitter, or 0 if
parameter=value pairs. transmission order is not constrained. The optional parameters go in
the SDP "a=fmtp" attribute by copying them directly from the MIME
media type string as a semicolon-separated list of parameter=value
pairs.
A sample SDP mapping for JPEG XS video is as follows: A sample SDP mapping for JPEG XS video is as follows:
m=video 30000 RTP/AVP 112 m=video 30000 RTP/AVP 112
a=rtpmap:112 jxsv/90000 a=rtpmap:112 jxsv/90000/1
a=fmtp:112 sampling=YCbCr-4:2:2; width=1920; height=1080; a=fmtp:112 sampling=YCbCr-4:2:2; width=1920; height=1080;
depth=10; colorimetry=BT709; TCS=SDR; depth=10; colorimetry=BT709; TCS=SDR;
RANGE=FULL; TP=2110TPNL RANGE=FULL; TP=2110TPNL
In this example, a JPEG XS RTP stream is being sent to UDP In this example, a JPEG XS RTP stream is being sent to UDP
destination port 30000, with an RTP dynamic payload type of 112 and a destination port 30000, with an RTP dynamic payload type of 112 and a
media clock rate of 90000 Hz. Note that the "a=fmtp:" line has been media clock rate of 90000 Hz. Note that the "a=fmtp:" line has been
wrapped to fit this page, and will be a single long line in the SDP wrapped to fit this page, and will be a single long line in the SDP
file. file.
6.2.3. Traffic shaping 6.2.3. Traffic shaping
The SDP object shall include the TP parameter (either 2110TPNL or The SDP object shall include the TP parameter (either 2110TPNL or
2110TPW as specified in Section 4.3) and may include the CMAX 2110TPW as specified in Section 4.4) and may include the CMAX
parameter as specified in SMPTE ST 2110-21 [SMPTE-ST2110-21]. parameter as specified in SMPTE ST 2110-21 [SMPTE-ST2110-21].
6.2.4. Offer/Answer Considerations 6.2.4. Offer/Answer Considerations
The following considerations apply when using SDP offer/answer The following considerations apply when using SDP offer/answer
procedures [RFC3264] to negotiate the use of the JPEG XS payload in procedures [RFC3264] to negotiate the use of the JPEG XS payload in
RTP: RTP:
o The "encode" parameter can be used for sendrecv, sendonly, and o The "encode" parameter can be used for sendrecv, sendonly, and
recvonly streams. Each encode type MUST use a separate payload recvonly streams. Each encode type MUST use a separate payload
skipping to change at page 18, line 51 skipping to change at page 22, line 51
Authors' Addresses Authors' Addresses
Sebastien Lugan Sebastien Lugan
intoPIX S.A. intoPIX S.A.
Rue Emile Francqui, 9 Rue Emile Francqui, 9
1435 Mont-Saint-Guibert 1435 Mont-Saint-Guibert
Belgium Belgium
Phone: +32 10 23 84 70 Phone: +32 10 23 84 70
Email: D313B41E@dynmail.crt1.net Email: rtp@intopix.com
URI: http://www.intopix.com URI: http://www.intopix.com
Gael Rouvroy Antonin Descampe
intoPIX S.A. intoPIX S.A.
Rue Emile Francqui, 9 Rue Emile Francqui, 9
1435 Mont-Saint-Guibert 1435 Mont-Saint-Guibert
Belgium Belgium
Phone: +32 10 23 84 70 Phone: +32 10 23 84 70
Email: g.rouvroy@intopix.com Email: a.descampe@intopix.com
URI: http://www.intopix.com URI: http://www.intopix.com
Antonin Descampe Corentin Damman
intoPIX S.A. intoPIX S.A.
Rue Emile Francqui, 9 Rue Emile Francqui, 9
1435 Mont-Saint-Guibert 1435 Mont-Saint-Guibert
Belgium Belgium
Phone: +32 10 23 84 70 Phone: +32 10 23 84 70
Email: a.descampe@intopix.com Email: c.damman@intopix.com
URI: http://www.intopix.com URI: http://www.intopix.com
Thomas Richter Thomas Richter
Fraunhofer IIS Fraunhofer IIS
Am Wolfsmantel 33 Am Wolfsmantel 33
91048 Erlangen 91048 Erlangen
Germany Germany
Phone: +49 9131 776 5126 Phone: +49 9131 776 5126
Email: thomas.richter@iis.fraunhofer.de Email: thomas.richter@iis.fraunhofer.de
 End of changes. 49 change blocks. 
192 lines changed or deleted 390 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/