[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00

RTCWEB Working Group                                           G. Garcia
Internet-Draft                                                    TokBox
Intended status: Informational                           August 05, 2013
Expires: February 06, 2014


          Simulcast and layered video coding support in WebRTC
           draft-garcia-simulcast-and-layered-video-webrtc-00

Abstract

   This document describes the use cases and requirements for simulcast
   and layered video coding support in WebRTC.  These techniques
   simplify the implementation of video stream adaptation to different
   participants in centralized conferencing solutions.  This document
   also includes a proposal to expose these capabilities in the existing
   PeerConnection API by defining new media constraint properties.

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on February 06, 2014.

Copyright Notice

   Copyright (c) 2013 IETF Trust and the persons identified as the
   document authors.  All rights reserved.












Garcia                  Expires February 06, 2014               [Page 1]


Internet-Draft     Simulcast and layered video coding        August 2013


   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
     1.1.  Browser support status  . . . . . . . . . . . . . . . . .   3
     1.2.  Requirements Language . . . . . . . . . . . . . . . . . .   3
   2.  Use-cases . . . . . . . . . . . . . . . . . . . . . . . . . .   3
     2.1.  Adaptation to devices with different capabilities . . . .   3
     2.2.  Adaptation to participants with different network
           conditions  . . . . . . . . . . . . . . . . . . . . . . .   3
     2.3.  Recording . . . . . . . . . . . . . . . . . . . . . . . .   4
     2.4.  Increasing video quality for active speaker . . . . . . .   4
   3.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   4
   4.  Proposed API  . . . . . . . . . . . . . . . . . . . . . . . .   5
     4.1.  Simulcasted streams . . . . . . . . . . . . . . . . . . .   5
     4.2.  Layered video coding  . . . . . . . . . . . . . . . . . .   5
     4.3.  Example . . . . . . . . . . . . . . . . . . . . . . . . .   6
   5.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .   7
   6.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .   7
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .   7
   8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   7
     8.1.  Normative References  . . . . . . . . . . . . . . . . . .   7
     8.2.  Informative References  . . . . . . . . . . . . . . . . .   7
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .   8

1.  Introduction

   Video conferencing using a central server is one of the typical use
   cases for real-time communication capabilities in browsers
   [I-D.ietf-rtcweb-use-cases-and-requirements].

   Most of today's multiparty videoconference solutions make use of
   centralized servers to reduce the bandwidth and CPU consumption in
   the endpoints.  Those servers receive streams from each participant
   and send the streams to rest of the participants, which usually have
   heterogeneous capabilities (screen size, CPU, bandwidth, etc.).  One
   of the biggest issues is how to perform the adaption to different
   participants' constraints with the minimum possible impact on video
   quality and server performance.



Garcia                  Expires February 06, 2014               [Page 2]


Internet-Draft     Simulcast and layered video coding        August 2013


   There are two approaches to adapt the streams to different
   destinations: one is transcoding (sometimes including mixing), and
   the other is switching between multiple streams or sub-streams
   received from the originator.  The first solution is computationally
   expensive and can degrade video quality.  The second solution makes a
   suboptimal use of network resources by sending redundant information,
   and in addition it is codec-specific.

   The requirements and proposed API in this document are based on
   existing JSEP API version and VP8 capabilities.  These are the
   technologies available in existing WebRTC browsers, but this proposal
   could be extended to other codecs or mapped to other APIs.

1.1.  Browser support status

   It is possible to use simulcast with existing WebRTC implementations.
   However, this requires the use of different PeerConnection objects,
   and all streams will have the same resolution.

   Multi-layer encoding is implemented and working in existing WebRTC
   browsers, and it has been tested in prototypes, but currently there
   is no way for developers to enable it.  In VP8 there is support for
   temporal scalability, while VP9 will include more advanced control
   and support for both temporal and spatial scalability.

1.2.  Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

2.  Use-cases

   The use cases envisioned for these new WebRTC capabilities are
   focused on centralized conferencing solutions.

2.1.  Adaptation to devices with different capabilities

   Some endpoints connected to a centralized conferencing server have
   small screens and do not need to receive high-resolution video, or
   the CPU power and battery consumption make it impossible to receive
   and decode high-resolution video in real-time.

   In this situation, it is desirable to send lower-resolution video to
   those endpoints.

2.2.  Adaptation to participants with different network conditions




Garcia                  Expires February 06, 2014               [Page 3]


Internet-Draft     Simulcast and layered video coding        August 2013


   Some endpoints connected to a centralized conferencing server do not
   have enough available bandwidth to receive high-quality video, while
   other endpoints have enough available bandwidth.

   In this situation is desirable to send lower-bitrate video to those
   endpoints.

2.3.  Recording

   A conferencing server implements recording and wants to record video
   in the highest quality possible, while forwarding it in lower quality
   to endpoints.

2.4.  Increasing video quality for active speaker

   A videoconference application shows the video of the active speaker
   in a larger size than videos of the other participants.

   It is desirable to increase the resolution and quality of that
   highlighted video stream, to maintain the perceived video quality.
   One possible implementation to increase the quality is to have a
   paused high-quality stream that resumes when voice activity is
   detected.

3.  Requirements

   This section contains the requirements for the API exposed in the
   browser, derived from the use-cases in Section 2.

   Requirements on how and when to enable scalable video coding:

   o  REQ-1.  It must be possible to enable and configure the scalable
      video coding before initiating a peer connection.

   o  REQ-2.  It must be possible to enable and configure the scalable
      video coding before answering a peer connection.

   o  REQ-3.  It must be possible to enable/disable and re-configure the
      scalable video coding to update a peer connection.

   Requirements on the parameters that needs to be configurable:

   o  REQ-5.  It must be possible to configure the number of simulcasted
      streams.

   o  REQ-6.  It must be possible to configure the minimum and maximum
      bitrate of each simulcasted stream.




Garcia                  Expires February 06, 2014               [Page 4]


Internet-Draft     Simulcast and layered video coding        August 2013


   o  REQ-7.  It must be possible to configure the resolution of each
      simulcasted stream.

   o  REQ-8.  It must be possible to configure the number of temporal
      layers (1 to 4).  This should be the only mandatory parameter when
      enabling temporal scalability.

   o  REQ-9.  It must be possible to configure the bitrate, frame rate
      decimation factor and membership of frames to layers for each
      temporal layer of the VP8 stream.

   Requirements regarding RTP usage:

   o  REQ-10.  Congestion control must be supported for all the
      simulcasted streams between the configured boundaries (min/max
      bitrate).

   o  REQ-11.  Transmission of simulcasted streams must be signaled and
      negotiated in the SDP and transmitted in RTP sessions, making use
      of existing standard attributes
      [I-D.westerlund-avtcore-multistream-and-simulcast].

   o  REQ-12.  Any endpoint should be prepared to receive VP8 multi-
      layered encoded video not requiring out of band negotiation in
      SDP.

   Non functional requirements:

   o  REQ-13.  The exposed API must be extensible to new codecs or new
      codec parameters.

4.  Proposed API

   The existing solution in the WebRTC API to modify settings of a
   PeerConnection is to use media constraints.  This section defines
   some new media constrains to enable and configure the usage of
   simulcasted and layered video streams.

4.1.  Simulcasted streams

   Simulcast capabilities are codec-agnostic and do not require new
   media constraints.  Existing media constrains for resolution, frame
   rate and bitrate can be reused, but the API needs to support
   receiving a list of them instead of just one.

4.2.  Layered video coding





Garcia                  Expires February 06, 2014               [Page 5]


Internet-Draft     Simulcast and layered video coding        August 2013


   Multi-layer capabilities are codec-dependent.  For VP8, these are the
   configuration parameters exposed in the codec, and that needs to be
   translated to media constraints (the descriptions are taken from VP8
   source code):

   o  tsNumberLayers: This value specifies the number of coding layers
      to be used.

   o  tsTargetBitrate: These values specify the target coding bitrate
      for each coding layer.

   o  tsRateDecimator: These values specify the frame rate decimation
      factors to apply to each layer.

   o  tsPeriodicity: This value specifies the length of the sequence
      that defines the membership of frames to layers.  For example, if
      tsPeriodicity=8 then frames are assigned to coding layers with a
      repeated sequence of length 8.

   o  tsLayerId: This array defines the membership of frames to coding
      layers.  For a 2-layer encoding that assigns even numbered frames
      to one layer (0) and odd numbered frames to a second layer (1)
      with tsPeriodicity=8, then tsLayerId = (0,1,0,1,0,1,0,1).

4.3.  Example

   Example of media constraints to request two simulcasted streams, the
   first one with four temporal layers and default bitrate and the
   second one with a single layer and fixed bitrate.

      {
        video: [{
            width: 640,
            height: 480,
            codecs: {
              vp8: { tsNumberLayers: 4 }
            }
          },
          {
            width: 320,
            height: 240,
            bitrate: { min: 100000, max: 100000 }
          }]
        }
      }






Garcia                  Expires February 06, 2014               [Page 6]


Internet-Draft     Simulcast and layered video coding        August 2013


5.  Acknowledgements

6.  IANA Considerations

   This memo includes no request to IANA.

7.  Security Considerations

   No security implications foreseen.

8.  References

8.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

8.2.  Informative References

   [I-D.ietf-rtcweb-use-cases-and-requirements]
              Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real-
              Time Communication Use-cases and Requirements", draft-
              ietf-rtcweb-use-cases-and-requirements-11 (work in
              progress), June 2013.

   [I-D.narten-iana-considerations-rfc2434bis]
              Narten, T. and H. Alvestrand, "Guidelines for Writing an
              IANA Considerations Section in RFCs", draft-narten-iana-
              considerations-rfc2434bis-09 (work in progress), March
              2008.

   [I-D.westerlund-avtcore-multistream-and-simulcast]
              Westerlund, M. and B. Burman, "RTP Multiple Stream
              Sessions and Simulcast", draft-westerlund-avtcore-
              multistream-and-simulcast-00 (work in progress), July
              2011.

   [RFC2629]  Rose, M., "Writing I-Ds and RFCs using XML", RFC 2629,
              June 1999.

   [VP8]      The WebM Project, "VP8 source code", 2013,
              <http://www.webmproject.org/docs/vp8-sdk/>.









Garcia                  Expires February 06, 2014               [Page 7]


Internet-Draft     Simulcast and layered video coding        August 2013


Author's Address

   Gustavo Garcia
   TokBox
   115 Stillman Street
   San Francisco, CA
   US

   Email: gustavo@tokbox.com










































Garcia                  Expires February 06, 2014               [Page 8]


Html markup produced by rfcmarkup 1.124, available from https://tools.ietf.org/tools/rfcmarkup/