[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: (draft-edwards-avt-rtp-jpeg2000) 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 RFC 5371

INTERNET-DRAFT                                             Eric Edwards
draft-ietf-avt-rtp-jpeg2000-03.txt                      Satoshi Futemma
                                                       Nobuyoshi Tomita
                                                       Eisaburo Itakura
                                                       Sony Corporation
                                                          June 20, 2003
                                                 Expires: December 2003


              RTP Payload Format for JPEG 2000 Video Streams

Status of this Memo

    This document is an Internet-Draft and is in subject to all
    provisions of Section 10 of RFC2026.

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups. Note that
    other groups may also distribute working documents as
    Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six
    months and may be updated, replaced, or obsoleted by other
    documents at any time. It is inappropriate to use Internet-Drafts
    as reference materials or to cite them other than as "work in
    progress."

    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/ietf/1id-abstracts.txt

    The list of Internet-Drafts Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html.

Abstract

    This document describes a payload format for transporting JPEG
    2000 video streams using RTP (Real-time Transport Protocol). JPEG
    2000 video streams are formed as a continuous series of JPEG 2000
    still images. This payload format will allow for scalability and
    robustness of JPEG 2000's potential to be maximized in streaming
    applications.


Table of Contents

    1.      Introduction ..........................................  2
    1.1     Conventions Used in this Document .....................  3
    2.      JPEG 2000 Video Features ..............................  4
    3.      Payload Design ........................................  4
    4.      Payload Format ........................................  4
    4.1     RTP fixed header usage ................................  4
    4.2     RTP Payload header format .............................  5
    5.      RTP Packetization .....................................  7
    5.1     Separation at arbitrary lengths .......................  9
    5.2     General JPEG 2000 RTP packet types .................... 10
    6.      Scalable Delivery and Priority field .................. 10

Edwards, et al.                                                [Page 1]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    6.1     Priority mapping table ................................ 11
    6.1.1   Default priority mapping .............................. 11
    6.1.2   User-defined priority table ........................... 12
    6.2     Sender's Actions  ..................................... 12
    6.3     Receiver's Action ..................................... 13
    7.      JPEG 2000 main header compensation .................... 13
    7.1     Sender processing ..................................... 14
    7.2     Receiver processing ................................... 14
    8.      Optional Payload Header ............................... 15
    9.      Security Consideration ................................ 16
    10.     Recommended Practices ................................. 16
    11.     IANA Consideration .................................... 17
    12.     Normative References .................................. 18
    13.     Informative References ................................ 18
    14.     Authors' Addresses .................................... 18
    15.     Intellectual Property Right Statement ................. 19
    16.     Full Copyright Statement .............................. 19


1. Introduction

    This document specifies payload formats for JPEG 2000 video
    streams over the Real-time Transport Protocol (RTP). JPEG 2000 is
    an ISO/IEC International Standard developed for next-generation
    still image encoding.  Its basic encoding technology is described
    in [1][2].

    Part 3 of the JPEG 2000 standard defines Motion JPEG 2000[3].
    However, this defines only the file format but not the
    transmission format for streaming on the Internet.  For this
    reason, it is necessary to define the RTP format for JPEG 2000
    video streams. JPEG 2000 supports many features over the current
    JPEG standard [1][4][5]:

        o Higher compression efficiency than JPEG with less visual
          loss especially at extreme compression ratios.

        o A single code stream that offers both lossy and superior
          lossless compression.

        o Transmission over noisy environments.

        o Progressive transmission by pixel accuracy and resolution.

        o Random code stream access and processing.

    First, the JPEG-2000 algorithm is briefly explained below. Fig. 1
    shows a block diagram of JPEG 2000 encoding method.

                                                     +-----+
                                                     | ROI |
                                                     +-----+
                                                        |
                                                        V

Edwards, et al.                                                [Page 2]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

                   +----------+   +----------+   +------------+
                   |DC, comp. |   | Wavelet  |   |            |
    raw image  ==> |transform-|==>|transform-|==>|Quantization|==+
                   |  ation   |   |  ation   |   |            |  |
                   +----------+   +----------+   +------------+  |
                                                                 |
                  +-----------+   +----------+   +------------+  |
                  |           |   |          |   |            |  |
     JPEG 2000 <==| Data      |<==|Arithmetic|<==|Coefficient |<=+
     code stream  | Ordering  |   |  coding  |   |bit modeling|
                  +-----------+   +----------+   +------------+

                  Fig. 1: Block diagram of the JPEG 2000 encoder


    Each color component or tile is transformed into wavelet
    coefficients.  The component or tile is sub-sampled into various
    levels usually vertically and horizontally from high frequencies
    (which contains all the sharp details) to the low frequencies
    (which contains all the flat areas.)  Quantization is performed on
    the coefficients within each subband.  The wavelet coefficient is
    divided by the quantization step size and the result is truncated.
    After quantization, code blocks are formed from within the
    precincts within the tiles.  Precincts are a finer separation than
    tiles and code blocks are the smallest separation of the image
    data.  Entropy coding is performed within each code block and
    arithmetically encoded by bit plane.  After the coefficients of
    all code blocks have been coded into a short bit stream, a header
    is added turning it into a packet.  The header has all the
    information needed to decompress the packet into code blocks.  A
    group of packets is called layers.

    For additional features in transmitting, a re-ordering of the
    formed packets is necessary.  The standard has four ways to
    transmit and decode a compressed image by: resolution, quality,
    position, or component.

    This is only to serve as an introduction to JPEG 2000 and to aid
    in understanding the rest of this document.  Further details of
    the encoder can be found in various texts on JPEG 2000 [1].

    To decompress a JPEG 2000 code stream, one would follow the
    reverse order of the encoding order, minus the quantization step.
    It is outside the scope of this document to describe in detail
    this procedure.  Please refer to various JPEG 2000 texts for
    details [1].

1.1 Conventions Used in this Document

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
    NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
    in this document are to be interpreted as described in RFC2119
    [6].


Edwards, et al.                                                [Page 3]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

2. JPEG 2000 Video Features

    JPEG 2000 video streams are formed as a continuous series of JPEG
    2000 still images so the above features of JPEG 2000 can be used
    effectively.  A JPEG 2000 video stream has the following merits:

    SNR is improved at a low bit rate.  The formation can be used as a
    video stream format at a low bit rate.

    This is a Full Intra format, which each frame is independently
    compressed has a low encoding and decoding delay.

    JPEG 2000 has flexible and accurate rate control.  This is
    suitable for traffic control and congestion control at the network
    transmission.

    JPEG 2000 can provide its own code stream error resilience markers
    to aid in code stream recovery.

3. Payload Design

    To provide a payload format that exploits the JPEG 2000 video
    stream, described in the previous section, the following must be
    taken into consideration:

    - Provisions for packet loss

        On the Internet, 5% packet loss is common and this percentage
        may sometimes come to 20% or more.  To split JPEG 2000 video
        streams into RTP packets, efficient packetization of the code
        stream is required to minimize the effects of disabled
        decoding due to missing code-blocks over error prone
        environments.  If the main header is lost in transmission, the
        decoding ability is lost.  Accordingly, a system to compensate
        for the loss of the main header as much as possible is
        required.

    - A packetizing scheme that exploits JPEG 2000 functionality.

        A packetizing scheme so that an image can be progressively
        transmitted and reconstructed progressively by the receiver
        using JPEG 2000 functionality.  Maximizing performance over
        various network conditions and various computing power of
        receiving platforms.

4. Payload Format

4.1 RTP fixed header usage

    For each RTP packet, the RTP fixed header is followed by the JPEG
    2000 payload header, which is followed by JPEG 2000 code stream.
    The RTP header fields that have a meaning specific to the JPEG
    2000 video are described as follows:


Edwards, et al.                                                [Page 4]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    Payload type (PT): The payload type is dynamically assigned by
      means outside the scope of this document. A payload type in the
      dynamic range shall be chosen by means of an out of band
      signaling protocol (e.g., RTSP, SIP, etc.)

    Marker bit (M): The marker bit of the RTP fixed header MUST be set
      to 1 on the last RTP packet of a video frame, and otherwise, it
      must be 0.  When transmission is performed by multiple RTP
      sessions, the bit is set in the last packet of the frame in each
      session.

    Timestamp: The RTP timestamp is in units of 90 KHz. The same
      timestamp must appear in each fragment of a given frame. The
      initial value of the timestamp is random to make known plaintext
      attacks on encryption more difficult, even if the source itself
      does not encrypt, as the packets may flow through a translator
      that does.

4.2 RTP Payload header format

    The RTP payload header format for JPEG 2000 video stream is as
    follows:

     0                  1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |X|E|MHF|mh_id|T|     priority    |            tile number      |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |     reserved    |                     fragment offset         |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

        Fig. 2:  RTP payload header format for JPEG 2000


    X : 1 bit

        Extension bit flag.  This bit MUST be set to 1 when a JPEG
        2000 optional payload header follows this header, the JPEG
        2000 payload header, otherwise it MUST be set to 0.  The
        details of optional payload headers are described in Section 8
        of thisdocument.

    E : 1 bit

        Enable bit flag.  If this bit is set to 1, it means
        "intelligent packetization" described in Section 5.2.  If E
        bit is 0, it means non-intelligent packetization" and a
        receiver MUST ignore any other payload header information
        other than extension bit flag and fragment offset.


    MHF (Main Header Flag) : 2 bits

        MHF shows whether the main header is packed into the RTP

Edwards, et al.                                                [Page 5]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

        packet or not.  When the main header exists in the RTP packet,
        the sender MUST set the first bit to 1, otherwise this field
        MUST set to 0.  If the first bit is 1, the second bit is
        valid, and if the last part of the main header is included
        (either whole or fragmented), the sender MUST set the second
        bit to 1. In other words, this field is either 3(=0b11) or
        2(=0b10) if the main header exists in the RTP packet,
        otherwise 0. Table of MHF usage is below:

        +----+-------------------------------------------------------+
        |MHF | Description                                           |
        +----+-------------------------------------------------------+
        | 00 | no main header is packed at all                       |
        | 01 | reserved for future use.                              |
        | 10 | the fragmented main header (not last part) is packed. |
        | 11 | a whole main header or the last part of the           |
        |    | fragmented main header is packed.                     |
        +----+-------------------------------------------------------+

                           Table 1: MHF usage values


        The receiver checks MHF to determine the main header range and
        may perform main header compensation described in Section 7 if
        the main header is lost.

    mh_id : 3 bits

        Main header identification value.  This is used for the JPEG
        2000 main header recovery.  The same mh_id is used as long as
        the coding parameters described in the main header remain
        unchanged.  The initial value of mh_id is random. Mh_id value
        must increase by 1 every time a new main header is
        transmitted.  Once the mh_id value is greater than 7, it must
        roll over and start at 1 again.  Usage of this header is
        described in Section 7 of this document. This field is only
        valid when E bit is 1.  If the E bit is 0, then this field
        SHOULD be zero.

    priority : 8 bits

        The priority field indicates the importance of the JPEG 2000
        packet included in the payload.  Typically, a higher priority
        is set in the packets containing the JPEG 2000 packets of the
        lower layers and the lower subbands.

    T (Tile flag) : 1 bit

        This field shows whether tile number field is valid or not:

        T=0 means that tile number field is valid and shows the tile
        number of the tile-part. The sender MUST set T flag to 0 when
        only one tile-part is packed into the RTP packet regardless
        whether it is a whole tile-part or a fragmentation of the

Edwards, et al.                                                [Page 6]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

        tile-part.

        T=1 means that tile number field is invalid. The sender MUST set
        T flag to 1 when the multiple whole tile-parts are packed into
        the RTP packet or there is no tile-part (in other words, only a
        main header) in the RTP packet.


    tile number : 16 bits

        The interpretation of this field is changed depending on the
        value of the T_flag.  When T=0, this field shows the tile
        number.  When T=1, tile number field is invalid. The sender
        SHOULD set tile number to 0, and the receiver MUST ignore this
        field.


    fragment offset : 24 bits

        This value must be set to the byte offset in the JPEG 2000
        data stream of this RTP packet's contents.

        JPEG 2000 frames are typically larger than underlying
        network's maximum transfer units (MTU), frames might be
        fragmented into several packets.  The fragment offset is the
        data offset in bytes of the current packet from the start of
        the JPEG 2000 code stream.  This field helps the receiver to
        reassemble JPEG 2000 code stream.

        To perform scalable video delivery by using multiple RTP
        sessions, the offset value from the first byte of the same
        frame is set for fragment offset.  Accordingly, in scalable
        video delivery using multiple RTP sessions, the fragment
        offset may not start with 0 in some RTP sessions even if the
        packet is the first one of the frame.

5. RTP Packetization

    Fig. 3 shows the construction of the JPEG 2000 code stream. The
    JPEG 2000 code stream consists of a main header beginning with the
    SOC marker, one or more tiles (only one tile for no tile
    division), and the EOC marker to indicate the end of the code
    steam.  Each tile consists of a tile-part header starts with the
    SOT marker and ending with the SOD marker, and a bit stream (a
    series of JPEG 2000 packets.)

          +--  +------------+
    Main  |    |     SOC    |  Required as the first marker.
    header|    +------------+
          |    |    main    |  Main header marker segments
          +--  +------------+
          |    |    SOT     |  Required at the beginning of each
    Tile- |    +------------+    tile-part header.
    part  |    |   T0,TP0   |  Tile 0, tile-part 0 header marker

Edwards, et al.                                                [Page 7]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    header|    +------------+    segments
          |    |    SOD     |  Required at the end of each tile-part
          +--  +------------+    header
               | bit stream |  Tile-part bit stream.
          +--  +------------+  Might include SOP and EPH
          |    |    SOT     |
    Tile- |    +------------+
    part  |    |   T1,TP0   |
    header|    +------------+
          |    |    SOD     |
          +--  +------------+
               | bit stream |
               +------------+
               |    EOC     |  Required as the last marker in the code
               +------------+  stream

              Fig. 3: Construction of the JPEG 2000 code stream


    The JPEG 2000 code stream consists of a main header, tile-part
    headers, and JPEG 2000 packets.  When we packetize the JPEG 2000
    code stream, these construction units from the code stream must be
    maintained.  Each RTP packet will consist of a main header,
    tile-part header, or JPEG 2000 packet.

    If the server does not understand JPEG 2000 code stream (i.e. the
    sender is not intelligent) it should pack JPEG 2000 code stream in
    the largest possible MTU data size for the RTP packet.  The sender
    must segment the JPEG 2000 code stream along arbitrary lengths
    into RTP sized packets for the receiver.  In this case, the E bit
    MUST be set to 0.  This type of packetization is called
    "non-intelligent packetization".

    If the sender understands JPEG 2000 code streams and can read the
    JPEG 2000 packets from the code stream.  (i.e. the sender is
    intelligent) This type of packetization is called "intelligent
    packetization".  JPEG 2000 packets should be packed into RTP
    payload packets in the following way:

    1. If the JPEG 2000 packets are smaller than the MTU size, the
       sender should put as many whole JPEG 2000 packets into a single
       RTP packet.  That is, the JPEG 2000 payload data should begin
       with either one of the SOC marker, SOT marker, or SOP marker
       (if it exists in the JPEG 2000 data stream).

    2. If the JPEG 2000 packets are larger than the MTU size, the
       sender should segment the JPEG 2000 packets at the largest
       possible MTU size but JPEG 2000 packets must not overlap.


    Regardless of the sender's capabilities, the receiver MUST be able
    to handle RTP packets of any size.

    If the sender does not fragment, any packets larger than the MTU

Edwards, et al.                                                [Page 8]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    size might be fragmented into multiple smaller IP packets than the
    MTU size by the IP layer.  If one fragmented IP packet is lost
    during transmission, it is recognized as a loss of the whole RTP
    packet because the receiving host might not be able to reassemble
    the RTP packet.

    The segmentation of the JPEG 2000 code stream into RTP packets
    must fit within the RTP payload size.

    For intelligent packetization, all packets SHOULD be 32 bit
    aligned.  If padding bits are required, then the padding bits MUST
    come at the end of the payload.  Any required padding bits MUST
    NOT appear between the header and the payload or at the beginning.

    In the following, all the possible packetization cases are
    described with diagrams.

5.1 Separation at arbitrary lengths

    In this case, a JPEG 2000 code stream is split into several
    fragments at arbitrary byte-position(Fig.4). The E bit MUST be set
    to 0 for this packetization type.

    +---+---+--------------------------+
    |RTP|PL |   jpeg 2000 codestream   |
    |hdr|hdr|   fragment (1)           |
    +---+---+--------------------------+
    +---+---+--------------------------+
    |RTP|PL |   jpeg 2000 codestream   |
    |hdr|hdr|   fragment (2)           |
    +---+---+--------------------------+
                ...
    +---+---+----------------------+---+
    |RTP|PL | jpeg 2000 codestream |EOC|
    |hdr|hdr| fragment (N)         |   |
    +---+---+----------------------+---+
    *PL hdr = payload header

                Fig. 4: Arbitrary length fragmentation.


    The E (Enable) bit flag in the payload header MUST be 0 for this
    packetization type.  All other fields except for the X bit and
    fragment offset field, in the payload header SHOULD be 0 and the
    receiver MUST ignore any other values when the enable bit is 0.

    Such RTP packetization scheme is not recommended from the
    standpoint of error resilience.  It is desirable to use it only in
    some limited environments shown below:


    - The sender finds it difficult to distinguish the main header,
      tile header, and JPEG 2000 packets from one another.  Such a
      situation is likely to occur when the sender has poor

Edwards, et al.                                                [Page 9]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

      computational power and there is no SOP marker in the JPEG 2000
      code stream.

    - The network environment is error free.

    - If the JPEG 2000 error resilience markers (TLM, PLM, PLT, PPM,
      and PPT markers) are present in the code stream.  Error
      resilience will be handled outside of RTP.  Its description is
      not within the scope of this document.  Using these markers may
      improve error resilience and recovery.  Producing JPEG 2000 bit
      streams with these markers is highly recommended in all cases.

5.2 General JPEG 2000 RTP packet types

    For the following packetization types, the E bit MUST be set to 1
    in all following cases.

    (1) JPEG 2000 main header (SOC marker) must come first after the
        payload header (just after the RTP payload header).  If a
        whole main header is packed into the RTP packet, the MHF_value
        must be 3 (=0b11). The tile-part header and jp2-packets MAY
        follow the main header in the same packet.  When only the main
        header is in the RTP packet, the T flag MUST be set to 1 and
        the tile number field is ignored.  The sender SHOULD set the
        tile number to 0x00, and the receiver MUST ignore this field.

    (2) If two or more tile-parts are packed into a single RTP packet,
        only whole tile-parts MUST be packed into the RTP packet.
        Segmented tile-packets MUST NOT be packed or spread over RTP
        several RTP packets.  When the multiple tile-parts exist in a
        single RTP packet, the T flag MUST be set to 1, which shows
        the tile number field is invald .

    (3) If one tile-part is packed into the RTP packet, the tile-part
        header, if any, MUST come first.  Note that the tile-part
        header just after the main header MAY either be packed with
        the main header, or be separated to another RTP packet.  In
        this case, T flag MUST be set to 0 and the tile number of the
        tile-part is set in the "tile number" field.  Jp2-packets MAY
        follow the tile-part header and may be packed into the same
        RTP packet.

    (4) If no headers of any kind are in the RTP packet, the T flag
        MUST be set to 0 and the tile number field MUST be set to the
        tile number which the jp2-packets belongs to.

    (5) If the main header, a tile-part header, or a jp2-packet is
        split into the multiple RTP packets, only one fragment SHALL
        be packed into an RTP packet.  If the main header is split,
        only the last fragment's MHF is 3 (=0b11), and the rest are
        2(=0b10) .  All other fragmented RTP packet's MHF value shall
        be 0.

6. Scalable Delivery and Priority field

Edwards, et al.                                               [Page 10]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003


    JPEG 2000 code stream has rich functionality built into it so
    decoders can easily handle scalable delivery or progressive
    transmission.  Progressive transmission that allows images to be
    reconstructed with increasing pixel accuracy or spatial resolution
    is essential for many applications.  This feature allows the
    reconstruction of images with different resolutions and pixel
    accuracy, as needed or desired, for different target devices.  The
    largest image source devices can provide a code stream that is
    easily processed for the smallest image display device.

    The JPEG 2000 packets contain all compressed image data from a
    specific layer, a specific component, a specific resolution level,
    and a specific precinct.  The order in which these packets are
    found in the code stream is called the "progression order".  The
    ordering of the packets can progress along four axes: layer,
    component, resolution level and precinct.

    Providing priority field to show importance of data contained in a
    given RTP packet can exploit JPEG 2000 progressive & scalable
    functions.

    The lower the priority value, the higher the priority.  In other
    words, the priority value 0 is the highest priority, and 255 is
    the lowest priority.  We define the priority value 0 and 255 as
    special priorities: 0 for the headers (the main header or
    tile-part header), and 255 for no priority.  When any headers (the
    main header or tile-part header) are packed into the RTP packet,
    the sender MUST set the priority value to 0.  When the sender will
    not use the priority field, the sender MUST set the priority value
    to 255 to inform the receiver that sender doesn't use the priority
    field.

6.1 Priority mapping table

    For the progression order, the priority value to be given to each
    JPEG 2000 packet is defined by the priority mapping table.  In
    principle, the priority mapping table is negotiated between the
    sender and the receiver through external protocols (such as: RTSP,
    SIP, etc), which not within the scope of this document.  However,
    in some environments such as a multicast videoconference
    environment, it might be difficult to negotiate the
    priority-mapping table between senders and receivers.  We define
    the default priority mapping for such a situation.  The receiver
    interprets the priority as a user-defined priority value only when
    the priority-mapping table has been negotiated and otherwise the
    receiver interprets as the default priority.

6.1.1 Default priority mapping

    The JPEG 2000 codestream is ordered in some progression order and
    the in most cases; the foremost jp2-packets are more important
    than the latter ones.  In the default priority table, jp2-packet
    number is used as a priority value.  Jp2-packet number is "packet

Edwards, et al.                                               [Page 11]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    sequence number" defined at SOP marker segments described in Annex
    A.8.1 [1]. The default priority values have a range from 1 to 254.
    If the number of packets is larger than 254, that is, a sequence
    number exceeds 254, the sender MUST set priority values of the
    following jp2-packets to 254.

6.1.2 User-defined priority table

    The user-defined priority table is freely defined by users, but
    priority value 0 and 255 MUST be used as a special priorities: 0
    for the headers and 255 for no priority.

    For example, in the LRCP order codestream with 3 layers and 3
    resolutions, the user-defined priority table can be defined below
    (the format is not significant).  It has 4 level priorities.

                priority 1: L=0,R=0, C=any, P=any
                priority 2: L=0,R=1-2, C=any, P=any
                priority 3: L=1,R=any, C=any, P=any
                priority 4: L=2,R=any, C=any, P=any

    As another example, the resolution-based priority table can be
    defined as below:

                Priority 1: R=0, L=0, C=any, P=any
                Priority 2: R=0, L=1-2, C=any, P=any
                Priority 3: R=1, L=any, C=any, P=any
                Priority 4: R=2, L=any, C=any, P=any


    As another example, the component-based priority table can be
    defined as below:

                Priority 1: C=0, L=0, R=0, P=any
                Priority 2: C=0, L=0, R=any, P=any
                            C=0, L=any, R=0, P=any
                Priority 3: C=1-2, L=any, R=any, P=any


    To change the priority-mapping table, a new priority-mapping table
    must be sent from the sender to the receiver as needed.

6.2 Sender's Actions

    Priority is given in accordance with the priority-mapping
    table. For RTP packets that only consist of a whole or fragmented
    main/tile header, the sender MUST set priority 0 when a
    priority-mapping table is used.  If a priority-mapping table is
    not used, the priority value must be 0xFF for the same RTP
    packets.

    When the several jp2-packets are packed into the same RTP packet,
    the priority values of these jp2-packets are sometimes
    different. In such a case, a sender MUST set the packet priority

Edwards, et al.                                               [Page 12]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    to the highest priority of all the ones inside the packet.  If the
    sender does not use any priority-mapping table, it MUST set 0xff
    in the priority field.

    The sender may transmit each priority using separate multiple RTP
    sessions defined by the priority value.  For example, different
    priority may be allocated to others in a multicast group.  The
    sender may also transmit all priority valued RTP packets using a
    single RTP session.

6.3 Receiver's Action

    Progressive transmission that allows images to be reconstructed
    with increasing pixel accuracy or spatial resolution is essential
    for many applications.  This feature allows the reconstruction of
    images with different resolutions and pixel accuracy, as needed or
    desired, for different target devices.  The image architecture
    provides for the efficient delivery of image data in many
    applications such as client/server applications.  The receiver
    should decode packets above a certain priority to obtain maximum
    performance depending on the receiver's platform.

    The receiver can determine on its own (using or not using the
    mapping table or other variables) the priority value level the RTP
    packets it should decode.

    For example, when a less powerful CPU is used or the terminal has
    only a low-resolution display, decoding only RTP packets below a
    certain priority permits obtaining optimal performance.

    If any high-priority RTP packet is not received when a packet loss
    occurs, frame(s) can be skipped because no significant visual loss
    may be perceived even if decoding can be successfully performed.

    When any uninterpretable or an unexpected priority is received,
    the receiver must interpret the packets as no priority
    (i.e. priority=0xFF).

7. JPEG 2000 main header compensation

    The JPEG 2000 image main header describes various encode
    parameters and the decoder decodes by using the parameters
    described in the main header.  If the RTP packet that contains the
    main header is lost, the corresponding JPEG 2000 code stream
    cannot and should not be decoded.  In an extremely rare case, if
    the main header has dropped and all the remainder JPEG 2000
    packets has been received successfully, the receiver cannot decode
    the frame without main header information.  Even when the main
    header is lost, it can be recovered to a certain level using the
    following method.

    A recovery of the main header that has been lost is very simple
    with this procedure.  In the case of JPEG 2000 video, it is common
    that encode parameters will not vary greatly from each successive

Edwards, et al.                                               [Page 13]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    frame. Even if the RTP packet including the main header of a frame
    has dropped, decoding processing may be performed by using the
    main header of the previous frame if this previous frame is
    already encoded by the same encode parameters.

    The mh_id field of the payload header is used to recognize whether
    the encoding parameters of the main header are the same as the
    encoding parameters of the previous frame. The same value is set
    in mh_id of the RTP packet in the same frame.  Mh_id and encode
    parameters are not associated with each other as 1:1 but they are
    used to recognize whether the encode parameters of the previous
    frame are the same or not.

    The mh_id field value SHOULD be saved from previous frames to be
    used to recover the current frame's main header, if lost.  If the
    mh_id of the current frame has the same value as the mh_id value
    of the previous frame, the previous frame's main header SHOULD be
    used to decode the current frame, in case of a lost header.

    The sender MUST increment mh_id when parameters in the header
    change and send a new main header accordingly.

    The receiver MAY use the md_id and MAY retain the header for such
    compensation.

7.1 Sender processing

    The sender must transmit RTP packets with the same mh_id value
    unless the encoder parameters are different from the previous
    frame.  The encode parameters are the fixed information marker
    segment (SIZ marker) and functional marker segments (COD, COC,
    RGN, QCD, QCC, and POC) specified in JPEG 2000 Part 1 Annex A [1].
    If the encode parameters have been changed, the sender
    transmitting RTP packets MUST increment the mh_id value by one.
    The initial mh_id value should be 1.  When the mh_id value exceeds
    7, the value MUST return to 1 again.

    If the md_id field is set to 0, the receiver MUST not save the
    main header and MUST NOT compensate for lost headers using the
    above method.

7.2 Receiver processing

    When the receiver has received the main header correctly, the RTP
    sequence number, the mh_id and main header should be saved except
    when the mh_id value is 0.  Only the last main header that was
    received correctly SHOULD be saved.  That is, if there has been a
    saved main header, the previous one is deleted and the new main
    header is saved.

    When the main header is not received, the receiver compares the
    current mh_id value (this mh_id can be known by receiving at least
    one RTP packet) with the saved mh_id value.  When the values are
    the same, decoding may be performed by using the saved main

Edwards, et al.                                               [Page 14]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    header.

    Knowing whether the main header is lost or not maybe difficult,
    especially when the main header is fragmented.

    In all cases, the main header will start with fragment offset = 0.
    In the case of fragmented main header, only the first fragment
    will have the fragment offset = 0.

8. Optional Payload Header

    When the extension bit of the JPEG 2000 payload header is 1, an
    optional payload header follows the payload header.  The JPEG 2000
    video stream payload comes after the optional payload header.  An
    optional payload header is intended for sending application
    specific data. As for receives, unrecognized optype should be
    ignored. The figure shows a general format of the optional payload
    header.

     0                  1                   2                  3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |    optype   |X|             length            |               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
    |                   option specific format .....                |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

        Fig. 5 : JPEG 2000 video stream optional payload header generic
                 format


    Optype : 7 bits

        Optype describes the optional payload header type.  Optypes
        0-63 are reserved as fixed, well-known mappings to be defined
        by future revisions of this document. Optypes 64-127 can be
        freely used for an application's own definition. If some
        options would be fully tested and widely used, they shall be
        registered with the Internet Assigned Number Authority (IANA).

    X : 1 bit

        Further extension bit. This must be set to 1 if another
        optional payload header follows this optional payload header;
        otherwise it must be set to 0.

        When the extension bit of the optional header is 1, another
        optional payload header MUST come immediately after this
        optional payload header.

    length : 16 bits

        This value must be the length of optional header in bytes. The
        receiver shall perform processing for the optional header when

Edwards, et al.                                               [Page 15]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

        the extension bit of the JPEG 2000 payload header is 1.

9. Security Consideration

    RTP packets using the payload format defined in this specification
    are subject to the security considerations discussed in the RTP
    specifications[7]. This implies that confidentiality of the media
    streams is achieved by encryption. Because the data compression
    used with this payload format is applied end-to-end, encryption
    may be performed on the compressed data so there is no conflict
    between the two operations.

    As this format transports jpeg2000 video streams, there are no
    significant security, confidentiality, or authentication concerns.

10. Recommended Practices

    As the JPEG 2000 coding standard is highly flexible, many
    different but compliant data streams can be produced and still be
    labeled as a JPEG 2000 data stream.

    The following is a set of recommendations set forth from our
    experience in developing JPEG 2000 and this payload
    specification. Implementations of this standard must handle all
    possibilities mentioned in this specification.  The following is a
    listing of items an implementation could optimize.

    Error Resilience Markers

        The use of error resilience markers in the JPEG 2000 data
        stream is highly recommended in all situations.  Error
        recovery with these markers is helpful to the decoder and save
        external resources.  Markers such as: RESET, RESTART, and
        ERTERM

    Packetization Ordering

        Packetization ordering is completely dependent on the client's
        capabilities.  Some orderings allow for less amount of
        distortion in the event of loss at the expense of memory
        storage and packet reordering.

    YCbCr Color space

        The YCbCr color space provides the greatest amount of
        compression in color with respect to the human visual
        system. When used with JPEG 2000, the usage of this color
        space can provide excellent visual results at extreme bit
        rates.

    Progression Ordering

        JPEG 2000 offers many different ways to order the final code
        stream to optimize the transfer with the presentation.  The

Edwards, et al.                                               [Page 16]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

        most useful ordering in our usage cases have been for layer
        progression and resolution progression ordering.

    Tiling and Packets

        JPEG 2000 packets are formed regardless of the encoding
        method. The encoder has little control over the size of these
        JPEG 2000 packets as they maybe large or small.

        Tiling splits the image up into smaller areas and each are
        encoded separately.  With tiles, the JPEG 2000 packet sizes
        are also reduced.  When using tiling, almost all JPEG 2000
        packet sizes are an acceptable size (i.e. smaller than the MTU
        size of most networks.)

        It is highly recommended that tiling be used so that
        packetization of JPEG 2000 packets for transport can be done
        simpler.

11. IANA Consideration

    This document defines a new RTP payload name and associated MIME
    type, JP2-VS. The MIME registration form for JPEG 2000 video
    stream is enclosed below:

    MIME media type name:  video

    MIME subtype name: JP2-VS

    Required parameters: none

    Optional parameters: none

    Encoding considerations:
        JPEG 2000 video stream can be transmitted
        with RTP as specified in RFC XXXX.

    Security considerations: see section 9 of RFC XXXX.

    Interoperability considerations:
        JPEG 2000 video stream is a sequence of JPEG 2000 still
        images. An implementation in compliant with [1] can decode and
        attempt to display the encoded JPEG 2000 video stream.

    Published specification: ISO/IEC 15444-1, RFC XXXX

    Applications which use this media type:
        video streaming and communication.

    Additional information: none

    Magic number(s): none

    File extension(s): none

Edwards, et al.                                               [Page 17]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003


    Macintosh File Type Code(s): none

    Person & email address to contact for further information:

        Eric Edwards
        Email: Eric.Edwards@am.sony.com

    Intended usage: COMMON

    Author/Change controller:
        Eric Edwards
        Email: Eric.Edwards@am.sony.com


12. Normative References

    [1] ISO/IEC JTC1/SC29, ISO/IEC 15444-1 "Information technology -
        JPEG 2000 image coding system - Part 1: Core coding system",
        July 2002.

    [6] S. Bradner, "Key words for use in RFCs to Indicate Requirement
        Levels", BCP14, RFC2119, March 1997.

    [7] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson,
        "RTP: A Transport Protocol for Real Time Applications", RFC
        1889, January 1996.

13. Informative References

    [2] ISO/IEC JTC1/SC29/WG1, "JPEG2000 Part I Final Committe Draft
        Version 1.0", http://www.jpeg.org/public/fcd15444-1.pdf, March
        2000.

    [3] ISO/IEC JTC1/SC29/WG1, "Motion JPEG 2000 Final Committee Draft
        1.0", http://www.jpeg.org/public/fcd15444-3.doc, March, 2001.

    [4] ISO/IEC JTC1/SC29/WG1, "JPEG2000 requirements and profiles
        version 6.3", draft in progress,
        http://www.jpeg.org/public/wg1n1803.pdf

    [5] Diego Santa-Cruz, Touradj Ebrahimi, Joel Askelof, Mathias
        Larsson and Charilaos Christopoulos, "JPEG 2000 still image
        coding versus other standards", In Proc. of SPIE's 45th annual
        meeting, Application of Digital Image Processing XXIII,
        vol.4115, pp.446-454, July 2000.

14. Authors' Addresses

    Eric Edwards
    Sony Corporation
    Media Processing Division
    Network & Software Technology Center of America
    3300 Zanker Road, MD: SJ2C4

Edwards, et al.                                               [Page 18]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    San Jose, CA 95134
    Phone: +1 408 955 6462
    Fax: +1 408 955 5724
    Email: Eric.Edwards@am.sony.com

    Satoshi Futemma/Nobuyoshi Tomita/Eisaburo Itakura
    Sony Corporation
    6-7-35 Kitashinagawa Shinagawa-ku
    Tokyo 141-0001 JAPAN
    Phone: +81 3 5448 3096
    Fax: +81 3 5448 4622
    Email: {satosi-f|n-tomita|itakura}@sm.sony.co.jp

15. Intellectual Property Right Statement

    The IETF takes no position regarding the validity or scope of any
    intellectual property or other rights that might be claimed to
    pertain to the implementation or use of the technology described
    in this document or the extent to which any license under such
    rights might or might not be available; neither does it represent
    that it has made any effort to identify any such rights.
    Information on the IETF's procedures with respect to rights in
    standards-track and standards-related documentation can be found
    in BCP-11.  Copies of claims of rights made available for
    publication and any assurances of licenses to be made available,
    or the result of an attempt made to obtain a general license or
    permission for the use of such proprietary rights by implementors
    or users of this specification can be obtained from the IETF
    Secretariat.

    The IETF invites any interested party to bring to its attention
    any copyrights, patents or patent applications, or other
    proprietary rights which may cover technology that may be required
    to practice this standard.  Please address the information to the
    IETF Executive Director.

    The IETF has been notified of intellectual property rights claimed
    in regard to some or all of the specification contained in this
    document.  For more information consult the online list of claimed
    rights.

16. Full Copyright Statement

    Copyright (C) The Internet Society (2003). All Rights Reserved.

    This document and translations of it may be copied and furnished
    to others, and derivative works that comment on or otherwise
    explain it or assist in its implmentation may be prepared, copied,
    published and distributed, in whole or in part, without
    restriction of any kind, provided that the above copyright notice
    and this paragraph are included on all such copies and derivative
    works.  However, this document itself may not be modified in any
    way, such as by removing the copyright notice or references to the
    Internet Society or other Internet organizations, except as needed

Edwards, et al.                                               [Page 19]

INTERNET-DRAFT    draft-ietf-avt-rtp-jpeg2000-03.txt      June 20, 2003

    for the purpose of developing Internet standards in which case the
    procedures for copyrights defined in the Internet Standards
    process must be followed, or as required to translate it into
    languages other than English.

    The limited permissions granted above are perpetual and will not
    be revoked by the Internet Society or its successors or assigns.

    This document and the information contained herein is provided on
    an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
    ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
    IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
    THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
    WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.










































Edwards, et al.                                               [Page 20]


Html markup produced by rfcmarkup 1.109, available from https://tools.ietf.org/tools/rfcmarkup/