[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits] [IPR]

Versions: (draft-romanow-clue-framework) 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Draft is active
In: MissingRef
CLUE WG                                            M. Duckworth, Ed.
Internet Draft                                                Polycom
Intended status: Informational                            A. Pepperell
Expires: June, 2013                                        Silverflare
                                                             S. Wenger
                                                                 Vidyo
                                                     December 24, 2012



               Framework for Telepresence Multi-Streams
                   draft-ietf-clue-framework-08.txt


Abstract

   This memo offers a framework for a protocol that enables devices
   in a telepresence conference to interoperate by specifying the
   relationships between multiple media streams.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current
   Internet-Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time.  It is inappropriate to use Internet-Drafts
   as reference material or to cite them other than as "work in
   progress."

   This Internet-Draft will expire on June 24, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document.  Code Components extracted from this
   document must include Simplified BSD License text as described in


Duckworth et. al.        Expires June 24 2013                 [Page 1]


Internet-Draft       CLUE Telepresence Framework        December 2012


   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Table of Contents




   1. Introduction...................................................3
   2. Terminology....................................................6
   3. Definitions....................................................6
   4. Overview of the Framework/Model................................9
   5. Spatial Relationships.........................................11
   6. Media Captures and Capture Scenes.............................12
      6.1. Media Captures...........................................12
         6.1.1. Media Capture Attributes............................12
      6.2. Capture Scene............................................15
         6.2.1. Capture scene attributes............................17
         6.2.2. Capture scene entry attributes......................18
      6.3. Simultaneous Transmission Set Constraints................19
   7. Encodings.....................................................20
      7.1. Individual Encodings.....................................21
      7.2. Encoding Group...........................................22
   8. Associating Media Captures with Encoding Groups...............24
   9. Consumer's Choice of Streams to Receive from the Provider.....25
      9.1. Local preference.........................................26
      9.2. Physical simultaneity restrictions.......................26
      9.3. Encoding and encoding group limits.......................26
      9.4. Message Flow.............................................27
   10. Extensibility................................................28
   11. Examples - Using the Framework...............................28
      11.1. Three screen endpoint media provider....................28
      11.2. Encoding Group Example..................................35
      11.3. The MCU Case............................................36
      11.4. Media Consumer Behavior.................................37
         11.4.1. One screen consumer................................37
         11.4.2. Two screen consumer configuring the example........38
         11.4.3. Three screen consumer configuring the example......38
   12. Acknowledgements.............................................39
   13. IANA Considerations..........................................39
   14. Security Considerations......................................39
   15. Changes Since Last Version...................................39
   16. Authors' Addresses...........................................42




Duckworth et. al.       Expires June 24, 2013                [Page 2]


Internet-Draft       CLUE Telepresence Framework        December 2012


1. Introduction

   Current telepresence systems, though based on open standards such
   as RTP [RFC3550] and SIP [RFC3261], cannot easily interoperate
   with each other.  A major factor limiting the interoperability of
   telepresence systems is the lack of a standardized way to describe
   and negotiate the use of the multiple streams of audio and video
   comprising the media flows.  This draft provides a framework for a
   protocol to enable interoperability by handling multiple streams
   in a standardized way.  It is intended to support the use cases
   described in draft-ietf-clue-telepresence-use-cases-02 and to meet
   the requirements in draft-ietf-clue-telepresence-requirements-01.

   Conceptually distinguished are Media Providers and Media
   Consumers.  A Media Provider provides Media in the form of RTP
   packets, a Media Consumer consumes those RTP packets.  Media
   Providers and Media Consumers can reside in Endpoints or in
   middleboxes such as Multipoint Control Units (MCUs).  A Media
   Provider in an Endpoint is usually associated with the generation
   of media for Media Captures; these Media Captures are typically
   sourced from cameras, microphones, and the like.  Similarly, the
   Media Consumer in an Endpoint is usually associated with
   Renderers, such as screens and loudspeakers.  In middleboxes,
   Media Providers and Consumers can have the form of outputs and
   inputs, respectively, of RTP mixers, RTP translators, and similar
   devices.  Typically, telepresence devices such as Endpoints and
   middleboxes would perform as both Media Providers and Media
   Consumers, the former being concerned with those devices'
   transmitted media and the latter with those devices' received
   media.  In a few circumstances, a CLUE Endpoint middlebox may
   include only Consumer or Provider functionality, such as recorder-
   type Consumers or webcam-type Providers.

   One initial motivation for this memo and its companion documents
   has been that Endpoints according to this memo can, and usually
   do, have multiple Media Captures and Media Renderers.  While
   previous system designs can deal with such a situation, what was
   missing was a mechanism that can associate the Media Captures with
   each other in space and time.  Further, due to the potentially
   large number of RTP flows required for a Multimedia Conference
   involving potentially many Endpoints, each of which can have many
   Media Captures and Media Renderers, a sensible system design is to
   multiplex multiple RTP media flows onto the same transport
   address, so to avoid using the port number as a multiplexing point
   and the associated shortcomings such as NAT/firewall traversal.


Duckworth et. al.       Expires June 24, 2013                [Page 3]


Internet-Draft       CLUE Telepresence Framework        December 2012


   While the actual mapping of those RTP flows to the header fields
   of the RTP packets is not subject of this specification, the large
   number of possible permutations of sensible options a Media
   Provider may make available to a Media Consumer makes a mechanism
   desirable that allows to narrow down the number of possible
   options that a SIP offer-answer exchange has to consider.  Such
   information is made available using protocol mechanisms specified
   in this memo and companion documents, although it should be
   stressed that its use in an implementation is optional.  Also,
   there are aspects of the control of both Endpoints and
   middleboxes/MCUs that dynamically change during the progress of a
   call, such as audio-level based screen switching, layout changes,
   and so on, which need to be conveyed.  Note that these control
   aspects are complementary to those specified in traditional SIP
   based conference management such as BFCP.  Finally, all this
   information needs to be conveyed, and the notion of support for it
   needs to be established.  This is done by the negotiation of a
   "CLUE channel", a data channel negotiated early during the
   initiation of a call.  An Endpoint or MCU that rejects the
   establishment of this data channel, by definition, is not
   supporting CLUE based mechanisms, whereas an Endpoint or MCU that
   accepts it is required to use it to the extent specified in this
   memo and its companion documents.

   A very brief outline of the call flow used by a simple system in
   compliance with this memo can be described as follows.

   An initial offer/answer exchange establishes a CLUE channel
   between two Endpoints.  With the establishment of that channel,
   the endpoints have consented to use the CLUE protocol mechanisms
   and have to adhere to them.

   Over this CLUE channel, the Provider in each Endpoint conveys its
   characteristics and capabilities as specified herein (which will
   typically not be sufficient to set up all media).  The Consumer in
   the Endpoint receives the information provided by the Provider,
   and can use it for two purposes.  First, it can, but is not
   necessarily required to, use the information provided to tailor
   the SDP it is going to send during the following SIP offer/answer
   exchange, and its reaction to SDP it receives in that step.  It is
   often a sensible implementation choice to do so, as the
   representation of the media information conveyed over the CLUE
   channel can dramatically cut down on the size of SDP messages used
   in the O/A exchange that follows.  Second, it takes note of the
   spatial relationship associated with the Media that are described.


Duckworth et. al.       Expires June 24, 2013                [Page 4]


Internet-Draft       CLUE Telepresence Framework        December 2012


   It is often sensible to take that spatial relationship into
   account when tailoring the SDP.

   This CLUE exchange is followed by an SDP offer answer exchange
   that not only establishes those aspects of the media that have not
   been "negotiated" over CLUE, but has also the side effect of
   setting up the media transmission itself, involving potentially
   security exchanges, ICE, and whatnot.  This step is plain vanilla
   SIP, with the exception that the SDP used herein, in most cases
   can (but not necessarily must) be considerably smaller than the
   SDP a system would typically need to exchange if there were no
   pre-established knowledge about the Provider and Consumer
   characteristics.

   During the lifetime of a call, further exchanges can occur over
   the CLUE channel.  In some cases, those further exchanges can be
   dealt with by Provider or Consumer without any other protocol
   activity.  For example, voice-activated screen switching, signaled
   over the CLUE channel, ought not to lead to heavy-handed
   mechanisms like SIP re-invites.  However, in other cases, after
   the CLUE negotiation an additional offer/answer exchange may
   become necessary.  For example, if both sides decide to upgrade
   the call from a single screen to a multi-screen call and more
   bandwidth is required for the additional video channels, that
   could require a new O/A exchange.

   Numerous optimizations may be possible, and are the implementer's
   choice.  For example, it may be sensible to establish one or more
   initial media channels during the initial offer/answer exchange,
   which would allow, for example, for a fast startup of audio.
   Depending on the system design, it may be possible to re-use this
   established channel using only CLUE mechanisms, thereby avoiding
   further offer/answer exchanges.

   One aspect of the protocol outlined herein and specified in
   normative detail in companion documents is that it makes available
   information regarding the Provider's capabilities to deliver
   Media, and attributes related to that media such as their spatial
   relationship, to the Media Consumer.  The operation of the
   Renderer inside the Consumer is unspecified in that it can choose
   to ignore some information provided by the Provider, and/or not
   render media streams available from the Provider (although it has
   to follow the CLUE protocol and, therefore, has to "accept" the
   Provider's information).  All CLUE protocol mechanisms are
   optional in the Consumer in the sense that, while the Consumer


Duckworth et. al.       Expires June 24, 2013                [Page 5]


Internet-Draft       CLUE Telepresence Framework        December 2012


   must be able to receive (and, potentially, gracefully acknowledge)
   CLUE messages, it is free to ignore the information provided
   therein.  Obviously, this is not a particularly sensible design
   choice.

   Legacy devices are defined here in as those Endpoints and MCUs
   that do not support the setup and use of the CLUE channel.  The
   notion of a device being a legacy device is established during the
   initial offer/answer exchange, in which the legacy device will not
   understand the offer for the CLUE channel and, therefore, reject
   it.  This is the indication for the CLUE-implementing Endpoint or
   MCU that the other side of the communication is not compliant with
   CLUE, and to fall back to whatever mechanism was used before the
   introduction of CLUE.

   As for the media, Provider and Consumer have an end-to-end
   communication relationship with respect to (RTP transported)
   media; and the mechanisms described herein and in companion
   documents do not change the aspects of setting up those RTP flows
   and sessions.  However, it should be noted that forms of RTP
   multiplexing of multiple RTP flows onto the same transport address
   are developed concurrently with the CLUE suite of specifications,
   and it is widely expected that most, if not all, Endpoints or MCUs
   supporting CLUE will also support those mechanisms.  Some design
   choices made in this memo reflect this coincidence in spec
   development timing.

2. Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
   NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
   in this document are to be interpreted as described in RFC 2119
   [RFC2119].

3. Definitions

   The terms defined below are used throughout this memo and
   companion documents and they are normative.  In order to easily
   identify the use of a defined term, those terms are capitalized.

   Audio Capture: Media Capture for audio.  Denoted as ACn.

   Camera-Left and Right: For media captures, camera-left and camera-
   right are from the point of view of a person observing the



Duckworth et. al.       Expires June 24, 2013                [Page 6]


Internet-Draft       CLUE Telepresence Framework        December 2012


   rendered media.  They are the opposite of stage-left and stage-
   right.

   Capture Device: A device that converts audio and video input into
   an electrical signal, in most cases to be fed into a media
   encoder.

   Cameras and microphones are examples for capture devices.

   Capture Encoding: A specific encoding of a media capture, to be
   sent by a media provider to a media consumer via RTP.

   Capture Scene: a structure representing the scene that is captured
   by a collection of capture devices.  A capture scene includes
   attributes and one or more capture scene entries, with each entry
   including one or more media captures.

   Capture Scene Entry: a list of media captures of the same media
   type that together form one way to represent the capture scene.

   Conference: used as defined in [RFC4353], A Framework for
   Conferencing within the Session Initiation Protocol (SIP).

   Individual Encoding: A variable with a set of attributes that
   describes the maximum values of a single audio or video capture
   encoding.  The attributes include: maximum bandwidth- and for
   video maximum macroblocks (for H.264), maximum width, maximum
   height, maximum frame rate.

   Encoding Group: A set of encoding parameters representing a total
   media encoding capability to be sub-divided across potentially
   multiple Individual Encodings.



   Endpoint: The logical point of final termination through
   receiving, decoding and rendering, and/or initiation through
   capturing, encoding, and sending of media streams.  An endpoint
   consists of one or more physical devices which source and sink
   media streams, and exactly one [RFC4353] Participant (which, in
   turn, includes exactly one SIP User Agent).  In contrast to an
   endpoint, an MCU may also send and receive media streams, but it
   is not the initiator nor the final terminator in the sense that
   Media is Captured or Rendered. Endpoints can be anything from
   multiscreen/multicamera rooms to handheld devices.


Duckworth et. al.       Expires June 24, 2013                [Page 7]


Internet-Draft       CLUE Telepresence Framework        December 2012


   Front: the portion of the room closest to the cameras.  In going
   towards back you move away from the cameras.

   MCU: Multipoint Control Unit (MCU) - a device that connects two or
   more endpoints together into one single multimedia conference
   [RFC5117].  An MCU includes an [RFC4353] Mixer.  [Edt. RFC4353 is
   tardy in requiring that media from the mixer be sent to EACH
   participant.  I think we have practical use cases where this is
   not the case.  But the bug (if it is one) is in 4353 and not
   herein.]



   Media: Any data that, after suitable encoding, can be conveyed
   over RTP, including audio, video or timed text.

   Media Capture: a source of Media, such as from one or more Capture
   Devices.  A Media Capture (MC) may be the source of one or more
   capture encodings.  A Media Capture may also be constructed from
   other Media streams.  A middle box can express Media Captures that
   it constructs from Media streams it receives.

   Media Consumer: an Endpoint or middle box that receives media
   streams

   Media Provider: an Endpoint or middle box that sends Media streams

   Model: a set of assumptions a telepresence system of a given
   vendor adheres to and expects the remote telepresence system(s)
   also to adhere to.

   Plane of Interest: The spatial plane containing the most relevant
   subject matter.

   Render: the process of generating a representation from a media,
   such as displayed motion video or sound emitted from loudspeakers.

   Simultaneous Transmission Set: a set of media captures that can be
   transmitted simultaneously from a Media Provider.

   Spatial Relation: The arrangement in space of two objects, in
   contrast to relation in time or other relationships.  See also
   Camera-Left and Right.




Duckworth et. al.       Expires June 24, 2013                [Page 8]


Internet-Draft       CLUE Telepresence Framework        December 2012


   Stage-Left and Right: For media captures, stage-left and stage-
   right are the opposite of camera-left and camera-right.  For the
   case of a person facing (and captured by) a camera, stage-left and
   stage-right are from the point of view of that person.

   Stream: a capture encoding sent from a media provider to a media
   consumer via RTP [RFC3550].

   Stream Characteristics: the media stream attributes commonly used
   in non-CLUE SIP/SDP environments (such as: media codec, bit rate,
   resolution, profile/level etc.) as well as CLUE specific
   attributes, such as the ID of a capture or a spatial location.

   Telepresence: an environment that gives non co-located users or
   user groups a feeling of (co-located) presence - the feeling that
   a Local user is in the same room with other Local users and the
   Remote parties.  The inclusion of Remote parties is achieved
   through multimedia communication including at least audio and
   video signals of high fidelity.

   Video Capture: Media Capture for video.  Denoted as VCn.

   Video composite: A single image that is formed from combining
   visual elements from separate sources.

4. Overview of the Framework/Model

   The CLUE framework specifies how multiple media streams are to be
   handled in a telepresence conference.

   The main goals include:

   o  Interoperability

   o  Extensibility

   o  Flexibility

   Interoperability is achieved by the media provider describing the
   relationships between media streams in constructs that are
   understood by the consumer, who can then render the media.
   Extensibility is achieved through abstractions and the generality
   of the model, making it easy to add new parameters.  Flexibility
   is achieved largely by having the consumer choose what content and



Duckworth et. al.       Expires June 24, 2013                [Page 9]


Internet-Draft       CLUE Telepresence Framework        December 2012


   format it wants to receive from what the provider is capable of
   sending.

   A transmitting endpoint or MCU describes specific aspects of the
   content of the media and the formatting of the media streams it
   can send (advertisement); and the receiving end responds to the
   provider by specifying which content and media streams it wants to
   receive (configuration).  The provider then transmits the asked
   for content in the specified streams.

   This advertisement and configuration occurs at call initiation but
   may also happen at any time throughout the conference, whenever
   there is a change in what the consumer wants or the provider can
   send.

   An endpoint or MCU typically acts as both provider and consumer at
   the same time, sending advertisements and sending configurations
   in response to receiving advertisements.  (It is possible to be
   just one or the other.)

   The data model is based around two main concepts: a capture and an
   encoding.  A media capture (MC), such as audio or video, describes
   the content a provider can send.  Media captures are described in
   terms of CLUE-defined attributes, such as spatial relationships
   and purpose of the capture.  Providers tell consumers which media
   captures they can provide, described in terms of the media capture
   attributes.

   A provider organizes its media captures that represent the same
   scene into capture scenes.  A consumer chooses which media
   captures it wants to receive according to the capture scenes sent
   by the provider.

   In addition, the provider sends the consumer a description of the
   individual encodings it can send in terms of the media attributes
   of the encodings, in particular, well-known audio and video
   parameters such as bandwidth, frame rate, macroblocks per second.

   The provider also specifies constraints on its ability to provide
   media, and the consumer must take these into account in choosing
   the content and capture encodings it wants.  Some constraints are
   due to the physical limitations of devices - for example, a camera
   may not be able to provide zoom and non-zoom views simultaneously.
   Other constraints are system based constraints, such as maximum
   bandwidth and maximum macroblocks/second.


Duckworth et. al.       Expires June 24, 2013               [Page 10]


Internet-Draft       CLUE Telepresence Framework        December 2012


   The following sections discuss these constructs and processes in
   detail, followed by use cases showing how the framework
   specification can be used.

5. Spatial Relationships

   In order for a consumer to perform a proper rendering, it is often
   necessary to provide spatial information about the streams it is
   receiving.  CLUE defines a coordinate system that allows media
   providers to describe the spatial relationships of their media
   captures to enable proper scaling and spatial rendering of their
   streams.  The coordinate system is based on a few principles:

   o  Simple systems which do not have multiple Media Captures to
      associate spatially need not use the coordinate model.

   o  Coordinates can either be in real, physical units
      (millimeters), have an unknown scale or have no physical scale.
      Systems which know their physical dimensions should always
      provide those real-world measurements.  Systems which don't
      know specific physical dimensions but still know relative
      distances should use 'unknown scale'.  'No scale' is intended
      to be used where Media Captures from different devices (with
      potentially different scales) will be forwarded alongside one
      another (e.g. in the case of a middle box).

      *  "millimeters" means the scale is in millimeters

      *  "Unknown" means the scale is not necessarily millimeters,
         but the scale is the same for every capture in the capture
         scene.

      *  "No Scale" means the scale could be different for each
         capture- an MCU provider that advertises two adjacent
         captures and picks sources (which can change quickly) from
         different endpoints might use this value; the scale could be
         different and changing for each capture.  But the areas of
         capture still represent a spatial relation between captures.

   o  The coordinate system is Cartesian X, Y, Z with the origin at a
      spot of the provider's choosing.  The provider must use the
      same coordinate system with same scale and origin for all
      coordinates within the same capture scene.




Duckworth et. al.       Expires June 24, 2013               [Page 11]


Internet-Draft       CLUE Telepresence Framework        December 2012


   The direction of increasing coordinate values is:
   X increases from camera left to camera right
   Y increases from front to back
   Z increases from low to high

6. Media Captures and Capture Scenes

   This section describes how media providers can describe the
   content of media to consumers.

6.1. Media Captures

   Media captures are the fundamental representations of streams that
   a device can transmit.  What a Media Capture actually represents
   is flexible:

   o  It can represent the immediate output of a physical source
      (e.g. camera, microphone) or 'synthetic' source (e.g. laptop
      computer, DVD player).

   o  It can represent the output of an audio mixer or video composer

   o  It can represent a concept such as 'the loudest speaker'

   o  It can represent a conceptual position such as 'the leftmost
      stream'

   To distinguish between multiple instances, video and audio
   captures are numbered such as: VC1, VC2 and AC1, AC2.  VC1 and VC2
   refer to two different video captures and AC1 and AC2 refer to two
   different audio captures.

   Each Media Capture can be associated with attributes to describe
   what it represents.

6.1.1. Media Capture Attributes

   Media Capture Attributes describe static information about the
   captures.  A provider uses the media capture attributes to
   describe the media captures to the consumer.  The consumer will
   select the captures it wants to receive.  Attributes are defined
   by a variable and its value.  The currently defined attributes and
   their values are:

   Content: {slides, speaker, sl, main, alt}


Duckworth et. al.       Expires June 24, 2013               [Page 12]


Internet-Draft       CLUE Telepresence Framework        December 2012


   A field with enumerated values which describes the role of the
   media capture and can be applied to any media type.  The
   enumerated values are defined by [RFC4796].  The values for this
   attribute are the same as the mediacnt values for the content
   attribute in [RFC4796].  This attribute can have multiple values,
   for example content={main, speaker}.

   Composed: {true, false}

   A field with a Boolean value which indicates whether or not the
   Media Capture is a mix (audio) or composition (video) of streams.

   This attribute is useful for a media consumer to avoid nesting a
   composed video capture into another composed capture or rendering.
   This attribute is not intended to describe the layout a media
   provider uses when composing video streams.

   Audio Channel Format: {mono, stereo} A field with enumerated
   values which describes the method of encoding used for audio.

   A value of 'mono' means the Audio Capture has one channel.

   A value of 'stereo' means the Audio Capture has two audio
   channels, left and right.

   This attribute applies only to Audio Captures.  A single stereo
   capture is different from two mono captures that have a left-right
   spatial relationship.  A stereo capture maps to a single RTP
   stream, while each mono audio capture maps to a separate RTP
   stream.

   Switched: {true, false}

   A field with a Boolean value which indicates whether or not the
   Media Capture represents the (dynamic) most appropriate subset of
   a 'whole'.  What is 'most appropriate' is up to the provider and
   could be the active speaker, a lecturer or a VIP.

   Point of Capture: {(X, Y, Z)}

   A field with a single Cartesian (X, Y, Z) point value which
   describes the spatial location, virtual or physical, of the
   capturing device (such as camera).




Duckworth et. al.       Expires June 24, 2013               [Page 13]


Internet-Draft       CLUE Telepresence Framework        December 2012


   When the Point of Capture attribute is specified, it must include
   X, Y and Z coordinates.  If the point of capture is not specified,
   it means the consumer should not assume anything about the spatial
   location of the capturing device.  Even if the provider specifies
   an area of capture attribute, it does not need to specify the
   point of capture.

   Point on Line of Capture: {(X,Y,Z)}

   A field with a single Cartesian (X, Y, Z) point value (virtual or
   physical) which describes a position in space of a second point on
   the axis of the capturing device; the first point being the Point
   of Capture (see above).  This point MUST lie between the Point of
   Capture and the Area of Capture.

   The Point on Line of Capture MUST be ignored if the Point of
   Capture is not present for this capture device.  When the Point on
   Line of Capture attribute is specified, it must include X, Y and Z
   coordinates.  These coordinates MUST NOT be identical to the Point
   of Capture coordinates.  If the Point on Line of Capture is not
   specified, no assumptions are made about the axis of the capturing
   device.

   Area of Capture:

   {bottom left(X1, Y1, Z1), bottom right(X2, Y2, Z2), top left(X3,
   Y3, Z3), top right(X4, Y4, Z4)}

   A field with a set of four (X, Y, Z) points as a value which
   describe the spatial location of what is being "captured".  By
   comparing the Area of Capture for different Media Captures within
   the same capture scene a consumer can determine the spatial
   relationships between them and render them correctly.

   The four points should be co-planar.  The four points form a
   quadrilateral, not necessarily a rectangle.

   The quadrilateral described by the four (X, Y, Z) points defines
   the plane of interest for the particular media capture.

   If the area of capture attribute is specified, it must include X,
   Y and Z coordinates for all four points.  If the area of capture
   is not specified, it means the media capture is not spatially
   related to any other media capture (but this can change in a
   subsequent provider advertisement).


Duckworth et. al.       Expires June 24, 2013               [Page 14]


Internet-Draft       CLUE Telepresence Framework        December 2012


   For a switched capture that switches between different sections
   within a larger area, the area of capture should use coordinates
   for the larger potential area.

   EncodingGroup: {<encodeGroupID value>}

   A field with a value equal to the encodeGroupID of the encoding
   group associated with the media capture.

   Max Capture Encodings: {unsigned integer}

   An optional attribute indicating the maximum number of capture
   encodings that can be simultaneously active for the media capture.
   If absent, this parameter defaults to 1.  The minimum value for
   this attribute is 1.  The number of simultaneous capture encodings
   is also limited by the restrictions of the encoding group for the
   media capture.

6.2. Capture Scene

   In order for a provider's individual media captures to be used
   effectively by a consumer, the provider organizes the media
   captures into capture scenes, with the structure and contents of
   these capture scenes being sent from the provider to the consumer.

   A capture scene is a structure representing the scene that is
   captured by a collection of capture devices.  A capture scene
   includes one or more capture scene entries, with each entry
   including one or more media captures.  A capture scene represents,
   for example, the video image of a group of people seated next to
   each other, along with the sound of their voices, which could be
   represented by some number of VCs and ACs in the capture scene
   entries.  A middle box may also express capture scenes that it
   constructs from media streams it receives.

   A provider may advertise multiple capture scenes or just a single
   capture scene.  A media provider might typically use one capture
   scene for main participant media and another capture scene for a
   computer generated presentation.  A capture scene may include more
   than one type of media.  For example, a capture scene can include
   several capture scene entries for video captures, and several
   capture scene entries for audio captures.

   A provider can express spatial relationships between media
   captures that are included in the same capture scene.  But there


Duckworth et. al.       Expires June 24, 2013               [Page 15]


Internet-Draft       CLUE Telepresence Framework        December 2012


   is no spatial relationship between media captures that are in
   different capture scenes.

   A media provider arranges media captures in a capture scene to
   help the media consumer choose which captures it wants.  The
   capture scene entries in a capture scene are different
   alternatives the provider is suggesting for representing the
   capture scene.  The media consumer can choose to receive all media
   captures from one capture scene entry for each media type (e.g.
   audio and video), or it can pick and choose media captures
   regardless of how the provider arranges them in capture scene
   entries.  Different capture scene entries of the same media type
   are not necessarily mutually exclusive alternatives.

   Media captures within the same capture scene entry must be of the
   same media type - it is not possible to mix audio and video
   captures in the same capture scene entry, for instance.  The
   provider must be capable of encoding and sending all media
   captures in a single entry simultaneously.  A consumer may decide
   to receive all the media captures in a single capture scene entry,
   but a consumer could also decide to receive just a subset of those
   captures.  A consumer can also decide to receive media captures
   from different capture scene entries.

   When a provider advertises a capture scene with multiple entries,
   it is essentially signaling that there are multiple
   representations of the same scene available.  In some cases, these
   multiple representations would typically be used simultaneously
   (for instance a "video entry" and an "audio entry").  In some
   cases the entries would conceptually be alternatives (for instance
   an entry consisting of 3 video captures versus an entry consisting
   of just a single video capture).  In this latter example, the
   provider would in the simple case end up providing to the consumer
   the entry containing the number of video captures that most
   closely matched the media consumer's number of display devices.

   The following is an example of 4 potential capture scene entries
   for an endpoint-style media provider:

   1.  (VC0, VC1, VC2) - left, center and right camera video captures

   2.  (VC3) - video capture associated with loudest room segment

   3.  (VC4) - video capture zoomed out view of all people in the
   room


Duckworth et. al.       Expires June 24, 2013               [Page 16]


Internet-Draft       CLUE Telepresence Framework        December 2012


   4.  (AC0) - main audio

   The first entry in this capture scene example is a list of video
   captures with a spatial relationship to each other.  Determination
   of the order of these captures (VC0, VC1 and VC2) for rendering
   purposes is accomplished through use of their Area of Capture
   attributes.  The second entry (VC3) and the third entry (VC4) are
   additional alternatives of how to capture the same room in
   different ways.  The inclusion of the audio capture in the same
   capture scene indicates that AC0 is associated with those video
   captures, meaning it comes from the same scene.  The audio should
   be rendered in conjunction with any rendered video captures from
   the same capture scene.

6.2.1. Capture scene attributes

   Attributes can be applied to capture scenes as well as to
   individual media captures.  Attributes specified at this level
   apply to all constituent media captures.

   Description attribute - list of {<description text>, <language
   tag>}

   The optional description attribute is a list of human readable
   text strings which describe the capture scene.  If there is more
   than one string in the list, then each string in the list should
   contain the same description, but in a different language.  A
   provider that advertises multiple capture scenes can provide
   descriptions for each of them.  This attribute can contain text in
   any number of languages.

   The language tag identifies the language of the corresponding
   description text.  The possible values for a language tag are the
   values of the 'Subtag' column for the "Type: language" entries in
   the "Language Subtag Registry" at [IANA-Lan] originally defined in
   [RFC5646].  A particular language tag value MUST NOT be used more
   than once in the description attribute list.

   Area of Scene attribute

   The area of scene attribute for a capture scene has the same
   format as the area of capture attribute for a media capture.  The
   area of scene is for the entire scene, which is captured by the
   one or more media captures in the capture scene entries.  If the
   provider does not specify the area of scene, but does specify


Duckworth et. al.       Expires June 24, 2013               [Page 17]


Internet-Draft       CLUE Telepresence Framework        December 2012


   areas of capture, then the consumer may assume the area of scene
   is greater than or equal to the outer extents of the individual
   areas of capture.

   Scale attribute

   An optional attribute indicating if the numbers used for area of
   scene, area of capture and point of capture are in terms of
   millimeters, unknown scale factor, or not any scale, as described
   in Section 5.  If any media captures have an area of capture
   attribute or point of capture attribute, then this scale attribute
   must also be defined.  The possible values for this attribute are:

      "millimeters"

      "unknown"

      "no scale"

6.2.2. Capture scene entry attributes

   Attributes can be applied to capture scene entries.  Attributes
   specified at this level apply to the capture scene entry as a
   whole.

   Scene-switch-policy: {site-switch, segment-switch}

   A media provider uses this scene-switch-policy attribute to
   indicate its support for different switching policies.  In the
   provider's advertisement, this attribute can have multiple values,
   which means the provider supports each of the indicated policies.
   The consumer, when it requests media captures from this capture
   scene entry, should also include this attribute but with only the
   single value (from among the values indicated by the provider)
   indicating the consumer's choice for which policy it wants the
   provider to use.  If the provider does not support any of these
   policies, it should omit this attribute.

   The "site-switch" policy means all captures are switched at the
   same time to keep captures from the same endpoint site together.
   Let's say the speaker is at site A and everyone else is at a
   "remote" site.

   When the room at site A shown, all the camera images from site A
   are forwarded to the remote sites.  Therefore at each receiving


Duckworth et. al.       Expires June 24, 2013               [Page 18]


Internet-Draft       CLUE Telepresence Framework        December 2012


   remote site, all the screens display camera images from site A.
   This can be used to preserve full size image display, and also
   provide full visual context of the displayed far end, site A. In
   site switching, there is a fixed relation between the cameras in
   each room and the displays in remote rooms.  The room or
   participants being shown is switched from time to time based on
   who is speaking or by manual control.

   The "segment-switch" policy means different captures can switch at
   different times, and can be coming from different endpoints.
   Still using site A as where the speaker is, and "remote" to refer
   to all the other sites, in segment switching, rather than sending
   all the images from site A, only the image containing the speaker
   at site A is shown.  The camera images of the current speaker and
   previous speakers (if any) are forwarded to the other sites in the
   conference.

   Therefore the screens in each site are usually displaying images
   from different remote sites - the current speaker at site A and
   the previous ones.  This strategy can be used to preserve full
   size image display, and also capture the non-verbal communication
   between the speakers.  In segment switching, the display depends
   on the activity in the remote rooms - generally, but not
   necessarily based on audio / speech detection.

6.3. Simultaneous Transmission Set Constraints

   The provider may have constraints or limitations on its ability to
   send media captures.  One type is caused by the physical
   limitations of capture mechanisms; these constraints are
   represented by a simultaneous transmission set.  The second type
   of limitation reflects the encoding resources available -
   bandwidth and macroblocks/second.  This type of constraint is
   captured by encoding groups, discussed below.

   An endpoint or MCU can send multiple captures simultaneously,
   however sometimes there are constraints that limit which captures
   can be sent simultaneously with other captures.  A device may not
   be able to be used in different ways at the same time.  Provider
   advertisements are made so that the consumer will choose one of
   several possible mutually exclusive usages of the device.  This
   type of constraint is expressed in a Simultaneous Transmission
   Set, which lists all the media captures that can be sent at the
   same time.  This is easier to show in an example.



Duckworth et. al.       Expires June 24, 2013               [Page 19]


Internet-Draft       CLUE Telepresence Framework        December 2012


   Consider the example of a room system where there are 3 cameras
   each of which can send a separate capture covering 2 persons each-
   VC0, VC1, VC2.  The middle camera can also zoom out and show all 6
   persons, VC3.  But the middle camera cannot be used in both modes
   at the same time - it has to either show the space where 2
   participants sit or the whole 6 seats, but not both at the same
   time.

   Simultaneous transmission sets are expressed as sets of the MCs
   that could physically be transmitted at the same time, (though it
   may not make sense to do so).  In this example the two
   simultaneous sets are shown in Table 1.  The consumer must make
   sure that it chooses one and not more of the mutually exclusive
   sets.  A consumer may choose any subset of the media captures in a
   simultaneous set, it does not have to choose all the captures in a
   simultaneous set if it does not want to receive all of them.

                           +-------------------+
                           | Simultaneous Sets |
                           +-------------------+
                           | {VC0, VC1, VC2}   |
                           | {VC0, VC3, VC2}   |
                           +-------------------+

                Table 1: Two Simultaneous Transmission Sets

   A media provider includes the simultaneous sets in its provider
   advertisement.  These simultaneous set constraints apply across
   all the captures scenes in the advertisement.  The simultaneous
   transmission sets MUST allow all the media captures in a
   particular capture scene entry to be used simultaneously.

7. Encodings

   We have considered how providers can describe the content of media
   to consumers.  We will now consider how the providers communicate
   information about their abilities to send streams.  We introduce
   two constructs - individual encodings and encoding groups.
   Consumers will then map the media captures they want onto the
   encodings with encoding parameters they want.  This process is
   then described.






Duckworth et. al.       Expires June 24, 2013               [Page 20]


Internet-Draft       CLUE Telepresence Framework        December 2012


7.1. Individual Encodings

   An individual encoding represents a way to encode a media capture
   to become a capture encoding, to be sent as an encoded media
   stream from the media provider to the media consumer.  An
   individual encoding has a set of parameters characterizing how the
   media is encoded.

   Different media types have different parameters, and different
   encoding algorithms may have different parameters.  An individual
   encoding can be assigned to only one capture encoding at a time.

   The parameters of an individual encoding represent the maximum
   values for certain aspects of the encoding.  A particular
   instantiation into a capture encoding might use lower values than
   these maximums.

   The following tables show the variables for audio and video
   encoding.

   +--------------+--------------------------------------------------
   --+
   | Name         | Description
   |
   +--------------+--------------------------------------------------
   --+
   | encodeID     | A unique identifier for the individual encoding
   |
   | maxBandwidth | Maximum number of bits per second
   |
   | maxH264Mbps  | Maximum number of macroblocks per second: ((width
   |
   |              | + 15) / 16) * ((height + 15) / 16) *
   |
   |              | framesPerSecond
   |
   | maxWidth     | Video resolution's maximum supported width,
   |
   |              | expressed in pixels
   |
   | maxHeight    | Video resolution's maximum supported height,
   |
   |              | expressed in pixels
   |
   | maxFrameRate | Maximum supported frame rate


Duckworth et. al.       Expires June 24, 2013               [Page 21]


Internet-Draft       CLUE Telepresence Framework        December 2012


   |
   +--------------+--------------------------------------------------
   --+

               Table 2: Individual Video Encoding Parameters

           +--------------+-----------------------------------+
           | Name         | Description                       |
           +--------------+-----------------------------------+
           | maxBandwidth | Maximum number of bits per second |
           +--------------+-----------------------------------+

               Table 3: Individual Audio Encoding Parameters

7.2. Encoding Group

   An encoding group includes a set of one or more individual
   encodings, plus some parameters that apply to the group as a
   whole.  By grouping multiple individual encodings together, an
   encoding group describes additional constraints on bandwidth and
   other parameters for the group.  Table 4 shows the parameters and
   individual encoding sets that are part of an encoding group.

























Duckworth et. al.       Expires June 24, 2013               [Page 22]


Internet-Draft       CLUE Telepresence Framework        December 2012


   +-------------------+---------------------------------------------
   --+
   | Name              | Description
   |
   +-------------------+---------------------------------------------
   --+
   | encodeGroupID     | A unique identifier for the encoding group
   |
   | maxGroupBandwidth | Maximum number of bits per second relating
   to |
   |                   | all encodings combined
   |
   | maxGroupH264Mbps  | Maximum number of macroblocks per second
   |
   |                   | relating to all video encodings combined
   |
   | videoEncodings[]  | Set of potential encodings (list of
   |
   |                   | encodeIDs)
   |
   | audioEncodings[]  | Set of potential encodings (list of
   |
   |                   | encodeIDs)
   |
   +-------------------+---------------------------------------------
   --+

                          Table 4: Encoding Group

   When the individual encodings in a group are instantiated into
   capture encodings, each capture encoding has a bandwidth that must
   be less than or equal to the maxBandwidth for the particular
   individual encoding.  The maxGroupBandwidth parameter gives the
   additional restriction that the sum of all the individual capture
   encoding bandwidths must be less than or equal to the
   maxGroupBandwidth value.

   Likewise, the sum of the macroblocks per second of each
   instantiated encoding in the group must not exceed the
   maxGroupH264Mbps value.

   The following diagram illustrates the structure of a media
   provider's Encoding Groups and their contents.




Duckworth et. al.       Expires June 24, 2013               [Page 23]


Internet-Draft       CLUE Telepresence Framework        December 2012


   ,-------------------------------------------------.
   |             Media Provider                      |
   |                                                 |
   |  ,--------------------------------------.       |
   |  | ,--------------------------------------.     |
   |  | | ,--------------------------------------.   |
   |  | | |          Encoding Group              |   |
   |  | | | ,-----------.                        |   |
   |  | | | |           | ,---------.            |   |
   |  | | | |           | |         | ,---------.|   |
   |  | | | | Encoding1 | |Encoding2| |Encoding3||   |
   |  `.| | |           | |         | `---------'|   |
   |    `.| `-----------' `---------'            |   |
   |      `--------------------------------------'   |
   `-------------------------------------------------'

                    Figure 1: Encoding Group Structure

   A media provider advertises one or more encoding groups.  Each
   encoding group includes one or more individual encodings.  Each
   individual encoding can represent a different way of encoding
   media.  For example one individual encoding may be 1080p60 video,
   another could be 720p30, with a third being CIF.


   While a typical 3 codec/display system might have one encoding
   group per "codec box", there are many possibilities for the number
   of encoding groups a provider may be able to offer and for the
   encoding values in each encoding group.

   There is no requirement for all encodings within an encoding group
   to be instantiated at once.

8. Associating Media Captures with Encoding Groups

   Every media capture is associated with an encoding group, which is
   used to instantiate that media capture into one or more capture
   encodings.  Each media capture has an encoding group attribute.
   The value of this attribute is the encodeGroupID for the encoding
   group with which it is associated.  More than one media capture
   may use the same encoding group.

   The maximum number of streams that can result from a particular
   encoding group constraint is equal to the number of individual
   encodings in the group.  The actual number of capture encodings


Duckworth et. al.       Expires June 24, 2013               [Page 24]


Internet-Draft       CLUE Telepresence Framework        December 2012


   used at any time may be less than this maximum.  Any of the media
   captures that use a particular encoding group can be encoded
   according to any of the individual encodings in the group.  If
   there are multiple individual encodings in the group, then the
   media consumer can configure the media provider to encode a single
   media capture into multiple different capture encodings at the
   same time, subject to the Max Capture Encodings constraint, with
   each capture encoding following the constraints of a different
   individual encoding.

   The Encoding Groups MUST allow all the media captures in a
   particular capture scene entry to be used simultaneously.

9. Consumer's Choice of Streams to Receive from the Provider

   After receiving the provider's advertised media captures and
   associated constraints, the consumer must choose which media
   captures it wishes to receive, and which individual encodings from
   the provider it wants to use to encode the captures.  Each media
   capture has an encoding group ID attribute which specifies which
   individual encodings are available to be used for that media
   capture.

   For each media capture the consumer wants to receive, it
   configures one or more of the encodings in that capture's encoding
   group.  The consumer does this by telling the provider the
   resolution, frame rate, bandwidth, etc. when asking for capture
   encodings for its chosen captures.  Upon receipt of this
   configuration command from the consumer, the provider generates a
   stream for each such configured capture encoding and sends those
   streams to the consumer.

   The consumer must have received at least one capture advertisement
   from the provider to be able to configure the provider's
   generation of media streams.

   The consumer is able to change its configuration of the provider's
   encodings any number of times during the call, either in response
   to a new capture advertisement from the provider or autonomously.
   The consumer need not send a new configure message to the provider
   when it receives a new capture advertisement from the provider
   unless the contents of the new capture advertisement cause the
   consumer's current configure message to become invalid.




Duckworth et. al.       Expires June 24, 2013               [Page 25]


Internet-Draft       CLUE Telepresence Framework        December 2012


   When choosing which streams to receive from the provider, and the
   encoding characteristics of those streams, the consumer needs to
   take several things into account: its local preference,
   simultaneity restrictions, and encoding limits.

9.1. Local preference

   A variety of local factors will influence the consumer's choice of
   streams to be received from the provider:

   o  if the consumer is an endpoint, it is likely that it would
      choose, where possible, to receive video and audio captures
      that match the number of display devices and audio system it
      has

   o  if the consumer is a middle box such as an MCU, it may choose
      to receive loudest speaker streams (in order to perform its own
      media composition) and avoid pre-composed video captures

   o  user choice (for instance, selection of a new layout) may
      result in a different set of media captures, or different
      encoding characteristics, being required by the consumer

9.2. Physical simultaneity restrictions

   There may be physical simultaneity constraints imposed by the
   provider that affect the provider's ability to simultaneously send
   all of the captures the consumer would wish to receive.  For
   instance, a middle box such as an MCU, when connected to a multi-
   camera room system, might prefer to receive both individual camera
   streams of the people present in the room and an overall view of
   the room from a single camera.  Some endpoint systems might be
   able to provide both of these sets of streams simultaneously,
   whereas others may not (if the overall room view were produced by
   changing the zoom level on the center camera, for instance).

9.3. Encoding and encoding group limits

   Each of the provider's encoding groups has limits on bandwidth and
   macroblocks per second, and the constituent potential encodings
   have limits on the bandwidth, macroblocks per second, video frame
   rate, and resolution that can be provided.  When choosing the
   media captures to be received from a provider, a consumer device
   must ensure that the encoding characteristics requested for each
   individual media capture fits within the capability of the


Duckworth et. al.       Expires June 24, 2013               [Page 26]


Internet-Draft       CLUE Telepresence Framework        December 2012


   encoding it is being configured to use, as well as ensuring that
   the combined encoding characteristics for media captures fit
   within the capabilities of their associated encoding groups.  In
   some cases, this could cause an otherwise "preferred" choice of
   capture encodings to be passed over in favour of different capture
   encodings - for instance, if a set of 3 media captures could only
   be provided at a low resolution then a 3 screen device could
   switch to favoring a single, higher quality, capture encoding.

9.4. Message Flow

   The following diagram shows the basic flow of messages between a
   media provider and a media consumer.  The usage of the "capture
   advertisement" and "configure encodings" message is described
   above. The consumer also sends its own capability message to the
   provider which may contain information about its own capabilities
   or restrictions.

   Diagram for Message Flow

            Media Consumer                         Media Provider
            --------------                         ------------
                  |                                     |
                  |----- Consumer Capability ---------->|
                  |                                     |
                  |                                     |
                  |<---- Capture advertisement ---------|
                  |                                     |
                  |                                     |
                  |------ Configure encodings --------->|
                  |                                     |

   In order for a maximally-capable provider to be able to advertise
   a manageable number of video captures to a consumer, there is a
   potential use for the consumer, at the start of CLUE, to be able
   to inform the provider of its capabilities.  One example here
   would be the video capture attribute set - a consumer could tell
   the provider the complete set of video capture attributes it is
   able to understand and so the provider would be able to reduce the
   capture scene it advertises to be tailored to the consumer.

   TBD - the content of the consumer capability message needs to be
   better defined.  The authors believe there is a need for this
   message, but have not worked out the details yet.



Duckworth et. al.       Expires June 24, 2013               [Page 27]


Internet-Draft       CLUE Telepresence Framework        December 2012


10. Extensibility

   One of the most important characteristics of the Framework is its
   extensibility.  Telepresence is a relatively new industry and
   while we can foresee certain directions, we also do not know
   everything about how it will develop.  The standard for
   interoperability and handling multiple streams must be future-
   proof. The framework itself is inherently extensible through
   expanding the data model types.  For example:

   o  Adding more types of media, such as telemetry, can done by
      defining additional types of captures in addition to audio and
      video.

   o  Adding new functionalities , such as 3-D, say, will require
      additional attributes describing the captures.

   o  Adding a new codecs, such as H.265, can be accomplished by
      defining new encoding variables.

   The infrastructure is designed to be extended rather than
   requiring new infrastructure elements.  Extension comes through
   adding to defined types.

   Assuming the implementation is in something like XML, adding data
   elements and attributes makes extensibility easy.

11. Examples - Using the Framework

   This section shows some examples in more detail how to use the
   framework to represent a typical case for telepresence rooms.
   First an endpoint is illustrated, then an MCU case is shown.

11.1. Three screen endpoint media provider

   Consider an endpoint with the following description:

   3 cameras, 3 displays, a 6 person table

   o  Each video device can provide one capture for each 1/3 section
      of the table

   o  A single capture representing the active speaker can be
      provided



Duckworth et. al.       Expires June 24, 2013               [Page 28]


Internet-Draft       CLUE Telepresence Framework        December 2012


   o  A single capture representing the active speaker with the other
      2 captures shown picture in picture within the stream can be
      provided

   o  A capture showing a zoomed out view of all 6 seats in the room
      can be provided

   The audio and video captures for this endpoint can be described as
   follows.

   Video Captures:

   o  VC0- (the camera-left camera stream), encoding group=EG0,
      content=main, switched=false

   o  VC1- (the center camera stream), encoding group=EG1,
      content=main, switched=false

   o  VC2- (the camera-right camera stream), encoding group=EG2,
      content=main, switched=false

   o  VC3- (the loudest panel stream), encoding group=EG1,
      content=main, switched=true

   o  VC4- (the loudest panel stream with PiPs), encoding group=EG1,
      content=main, composed=true, switched=true

   o  VC5- (the zoomed out view of all people in the room), encoding
      group=EG1, content=main, composed=false, switched=false

   o  VC6- (presentation stream), encoding group=EG1, content=slides,
      switched=false

   The following diagram is a top view of the room with 3 cameras, 3
   displays, and 6 seats.  Each camera is capturing 2 people.  The
   six seats are not all in a straight line.











Duckworth et. al.       Expires June 24, 2013               [Page 29]


Internet-Draft       CLUE Telepresence Framework        December 2012


      ,-. D
     (   )`--.__        +---+
      `-' /     `--.__  |   |
    ,-.  |            `-.._ |_-+Camera 2 (VC2)
   (   ).'        ___..-+-''`+-+
    `-' |_...---''      |   |
    ,-.c+-..__          +---+
   (   )|     ``--..__  |   |
    `-' |             ``+-..|_-+Camera 1 (VC1)
    ,-. |            __..--'|+-+
   (   )|     __..--'   |   |
    `-'b|..--'          +---+
    ,-. |``---..___     |   |
   (   )\          ```--..._|_-+Camera 0 (VC0)
    `-'  \             _..-''`-+
     ,-. \      __.--'' |   |
    (   ) |..-''        +---+
     `-' a


   The two points labeled b and c are intended to be at the midpoint
   between the seating positions, and where the fields of view of the
   cameras intersect.

   The plane of interest for VC0 is a vertical plane that intersects
   points 'a' and 'b'.

   The plane of interest for VC1 intersects points 'b' and 'c'. The
   plane of interest for VC2 intersects points 'c' and 'd'.

   This example uses an area scale of millimeters.

   Areas of capture:

       bottom left    bottom right  top left         top right
   VC0 (-2011,2850,0) (-673,3000,0) (-2011,2850,757) (-673,3000,757)
   VC1 ( -673,3000,0) ( 673,3000,0) ( -673,3000,757) ( 673,3000,757)
   VC2 (  673,3000,0) (2011,2850,0) (  673,3000,757) (2011,3000,757)
   VC3 (-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
   VC4 (-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
   VC5 (-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
   VC6 none

   Points of capture:
   VC0 (-1678,0,800)


Duckworth et. al.       Expires June 24, 2013               [Page 30]


Internet-Draft       CLUE Telepresence Framework        December 2012


   VC1 (0,0,800)
   VC2 (1678,0,800)
   VC3 none
   VC4 none
   VC5 (0,0,800)
   VC6 none

   In this example, the right edge of the VC0 area lines up with the
   left edge of the VC1 area.  It doesn't have to be this way.  There
   could be a gap or an overlap.  One additional thing to note for
   this example is the distance from a to b is equal to the distance
   from b to c and the distance from c to d.  All these distances are
   1346 mm. This is the planar width of each area of capture for VC0,
   VC1, and VC2.

   Note the text in parentheses (e.g. "the camera-left camera
   stream") is not explicitly part of the model, it is just
   explanatory text for this example, and is not included in the
   model with the media captures and attributes.  Also, the
   "composed" boolean attribute doesn't say anything about how a
   capture is composed, so the media consumer can't tell based on
   this attribute that VC4 is composed of a "loudest panel with
   PiPs".

   Audio Captures:

   o  AC0 (camera-left), encoding group=EG3, content=main, channel
      format=mono

   o  AC1 (camera-right), encoding group=EG3, content=main, channel
      format=mono

   o  AC2 (center) encoding group=EG3, content=main, channel
      format=mono

   o  AC3 being a simple pre-mixed audio stream from the room (mono),
      encoding group=EG3, content=main, channel format=mono

   o  AC4 audio stream associated with the presentation video (mono)
      encoding group=EG3, content=slides, channel format=mono

   Areas of capture:

       bottom left    bottom right  top left         top right



Duckworth et. al.       Expires June 24, 2013               [Page 31]


Internet-Draft       CLUE Telepresence Framework        December 2012


   AC0 (-2011,2850,0) (-673,3000,0) (-2011,2850,757) (-673,3000,757)
   AC1 (  673,3000,0) (2011,2850,0) (  673,3000,757) (2011,3000,757)
   AC2 ( -673,3000,0) ( 673,3000,0) ( -673,3000,757) ( 673,3000,757)
   AC3 (-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
   AC4 none

   The physical simultaneity information is:

      Simultaneous transmission set #1 {VC0, VC1, VC2, VC3, VC4, VC6}

      Simultaneous transmission set #2 {VC0, VC2, VC5, VC6}

   This constraint indicates it is not possible to use all the VCs at
   the same time.  VC5 can not be used at the same time as VC1 or VC3
   or VC4.  Also, using every member in the set simultaneously may
   not make sense - for example VC3(loudest) and VC4 (loudest with
   PIP).  (In addition, there are encoding constraints that make
   choosing all of the VCs in a set impossible.  VC1, VC3, VC4, VC5,
   VC6 all use EG1 and EG1 has only 3 ENCs.  This constraint shows up
   in the encoding groups, not in the simultaneous transmission
   sets.)

   In this example there are no restrictions on which audio captures
   can be sent simultaneously.

   Encoding Groups:

   This example has three encoding groups associated with the video
   captures.  Each group can have 3 encodings, but with each
   potential encoding having a progressively lower specification.  In
   this example, 1080p60 transmission is possible (as ENC0 has a
   maxMbps value compatible with that) as long as it is the only
   active encoding in the group(as maxMbps for the entire encoding
   group is also 489600).  Significantly, as up to 3 encodings are
   available per group, it is possible to transmit some video
   captures simultaneously that are not in the same entry in the
   capture scene.  For example VC1 and VC3 at the same time.

   It is also possible to transmit multiple capture encodings of a
   single video capture.  For example VC0 can be encoded using ENC0
   and ENC1 at the same time, as long as the encoding parameters
   satisfy the constraints of ENC0, ENC1, and EG0, such as one at
   1080p30 and one at 720p30.




Duckworth et. al.       Expires June 24, 2013               [Page 32]


Internet-Draft       CLUE Telepresence Framework        December 2012


   encodeGroupID=EG0, maxGroupH264Mbps=489600,
   maxGroupBandwidth=6000000
       encodeID=ENC0, maxWidth=1920, maxHeight=1088, maxFrameRate=60,
                      maxH264Mbps=489600, maxBandwidth=4000000
       encodeID=ENC1, maxWidth=1280, maxHeight=720, maxFrameRate=30,
                      maxH264Mbps=108000, maxBandwidth=4000000
       encodeID=ENC2, maxWidth=960, maxHeight=544, maxFrameRate=30,
                      maxH264Mbps=61200, maxBandwidth=4000000
   encodeGroupID=EG1 maxGroupH264Mbps=489600
   maxGroupBandwidth=6000000
       encodeID=ENC3, maxWidth=1920, maxHeight=1088, maxFrameRate=60,
                      maxH264Mbps=489600, maxBandwidth=4000000
       encodeID=ENC4, maxWidth=1280, maxHeight=720, maxFrameRate=30,
                      maxH264Mbps=108000, maxBandwidth=4000000
       encodeID=ENC5, maxWidth=960, maxHeight=544, maxFrameRate=30,
                      maxH264Mbps=61200, maxBandwidth=4000000
   encodeGroupID=EG2 maxGroupH264Mbps=489600
   maxGroupBandwidth=6000000
       encodeID=ENC6, maxWidth=1920, maxHeight=1088, maxFrameRate=60,
                      maxH264Mbps=489600, maxBandwidth=4000000
       encodeID=ENC7, maxWidth=1280, maxHeight=720, maxFrameRate=30,
                      maxH264Mbps=108000, maxBandwidth=4000000
       encodeID=ENC8, maxWidth=960, maxHeight=544, maxFrameRate=30,
                      maxH264Mbps=61200, maxBandwidth=4000000

                Figure 2: Example Encoding Groups for Video

   For audio, there are five potential encodings available, so all
   five audio captures can be encoded at the same time.

   encodeGroupID=EG3, maxGroupH264Mbps=0, maxGroupBandwidth=320000
       encodeID=ENC9, maxBandwidth=64000
       encodeID=ENC10, maxBandwidth=64000
       encodeID=ENC11, maxBandwidth=64000
       encodeID=ENC12, maxBandwidth=64000
       encodeID=ENC13, maxBandwidth=64000

                Figure 3: Example Encoding Group for Audio

   Capture Scenes:

   The following table represents the capture scenes for this
   provider. Recall that a capture scene is composed of alternative
   capture scene entries covering the same scene.  Capture Scene #1



Duckworth et. al.       Expires June 24, 2013               [Page 33]


Internet-Draft       CLUE Telepresence Framework        December 2012


   is for the main people captures, and Capture Scene #2 is for
   presentation.

   Each row in the table is a separate entry in the capture scene

                           +------------------+
                           | Capture Scene #1 |
                           +------------------+
                           | VC0, VC1, VC2    |
                           | VC3              |
                           | VC4              |
                           | VC5              |
                           | AC0, AC1, AC2    |
                           | AC3              |
                           +------------------+

                           +------------------+
                           | Capture Scene #2 |
                           +------------------+
                           | VC6              |
                           | AC4              |
                           +------------------+

   Different capture scenes are unique to each other, non-
   overlapping. A consumer can choose an entry from each capture
   scene.  In this case the three captures VC0, VC1, and VC2 are one
   way of representing the video from the endpoint.  These three
   captures should appear adjacent next to each other.
   Alternatively, another way of representing the Capture Scene is
   with the capture VC3, which automatically shows the person who is
   talking.  Similarly for the VC4 and VC5 alternatives.

   As in the video case, the different entries of audio in Capture
   Scene #1 represent the "same thing", in that one way to receive
   the audio is with the 3 audio captures (AC0, AC1, AC2), and
   another way is with the mixed AC3.  The Media Consumer can choose
   an audio capture entry it is capable of receiving.

   The spatial ordering is understood by the media capture attributes
   area and point of capture.

   A Media Consumer would likely want to choose a capture scene entry
   to receive based in part on how many streams it can simultaneously
   receive.  A consumer that can receive three people streams would
   probably prefer to receive the first entry of Capture Scene #1


Duckworth et. al.       Expires June 24, 2013               [Page 34]


Internet-Draft       CLUE Telepresence Framework        December 2012


   (VC0, VC1, VC2) and not receive the other entries.  A consumer
   that can receive only one people stream would probably choose one
   of the other entries.

   If the consumer can receive a presentation stream too, it would
   also choose to receive the only entry from Capture Scene #2 (VC6).

11.2. Encoding Group Example

   This is an example of an encoding group to illustrate how it can
   express dependencies between encodings.

   encodeGroupID=EG0, maxGroupH264Mbps=489600,
   maxGroupBandwidth=6000000
       encodeID=VIDENC0, maxWidth=1920, maxHeight=1088,
   maxFrameRate=60,
                         maxH264Mbps=244800, maxBandwidth=4000000
       encodeID=VIDENC1, maxWidth=1920, maxHeight=1088,
   maxFrameRate=60,
                         maxH264Mbps=244800, maxBandwidth=4000000
       encodeID=AUDENC0, maxBandwidth=96000
       encodeID=AUDENC1, maxBandwidth=96000
       encodeID=AUDENC2, maxBandwidth=96000

   Here, the encoding group is EG0.  It can transmit up to two
   1080p30 capture encodings (Mbps for 1080p = 244800), but it is
   capable of transmitting a maxFrameRate of 60 frames per second
   (fps).  To achieve the maximum resolution (1920 x 1088) the frame
   rate is limited to 30 fps.  However 60 fps can be achieved at a
   lower resolution if required by the consumer.  Although the
   encoding group is capable of transmitting up to 6Mbit/s, no
   individual video encoding can exceed 4Mbit/s.

   This encoding group also allows up to 3 audio encodings, AUDENC<0-
   2>. It is not required that audio and video encodings reside
   within the same encoding group, but if so then the group's overall
   maxBandwidth value is a limit on the sum of all audio and video
   encodings configured by the consumer.  A system that does not wish
   or need to combine bandwidth limitations in this way should
   instead use separate encoding groups for audio and video in order
   for the bandwidth limitations on audio and video to not interact.

   Audio and video can be expressed in separate encoding groups, as
   in this illustration.



Duckworth et. al.       Expires June 24, 2013               [Page 35]


Internet-Draft       CLUE Telepresence Framework        December 2012


   encodeGroupID=EG0, maxGroupH264Mbps=489600,
   maxGroupBandwidth=6000000
       encodeID=VIDENC0, maxWidth=1920, maxHeight=1088,
   maxFrameRate=60,
                         maxH264Mbps=244800, maxBandwidth=4000000
       encodeID=VIDENC1, maxWidth=1920, maxHeight=1088,
   maxFrameRate=60,
                         maxH264Mbps=244800, maxBandwidth=4000000
   encodeGroupID=EG1, maxGroupH264Mbps=0, maxGroupBandwidth=500000
       encodeID=AUDENC0, maxBandwidth=96000
       encodeID=AUDENC1, maxBandwidth=96000
       encodeID=AUDENC2, maxBandwidth=96000

11.3. The MCU Case

   This section shows how an MCU might express its Capture Scenes,
   intending to offer different choices for consumers that can handle
   different numbers of streams.  A single audio capture stream is
   provided for all single and multi-screen configurations that can
   be associated (e.g. lip-synced) with any combination of video
   captures at the consumer.

   +--------------------+--------------------------------------------
   -+
   | Capture Scene #1   | note
   |
   +--------------------+--------------------------------------------
   -+
   | VC0                | video capture for single screen consumer
   |
   | VC1, VC2           | video capture for 2 screen consumer
   |
   | VC3, VC4, VC5      | video capture for 3 screen consumer
   |
   | VC6, VC7, VC8, VC9 | video capture for 4 screen consumer
   |
   | AC0                | audio capture representing all participants
   |
   +--------------------+--------------------------------------------
   -+

   If / when a presentation stream becomes active within the
   conference the MCU might re-advertise the available media as:




Duckworth et. al.       Expires June 24, 2013               [Page 36]


Internet-Draft       CLUE Telepresence Framework        December 2012


        +------------------+--------------------------------------+
        | Capture Scene #2 | note                                 |
        +------------------+--------------------------------------+
        | VC10             | video capture for presentation       |
        | AC1              | presentation audio to accompany VC10 |
        +------------------+--------------------------------------+

11.4. Media Consumer Behavior

   This section gives an example of how a media consumer might behave
   when deciding how to request streams from the three screen
   endpoint described in the previous section.

   The receive side of a call needs to balance its requirements,
   based on number of screens and speakers, its decoding capabilities
   and available bandwidth, and the provider's capabilities in order
   to optimally configure the provider's streams.  Typically it would
   want to receive and decode media from each capture scene
   advertised by th provider.

   A sane, basic, algorithm might be for the consumer to go through
   eac capture scene in turn and find the collection of video
   captures that best matches the number of screens it has (this
   might include consideration of screens dedicated to presentation
   video display rather than "people" video) and then decide between
   alternative entries in the video capture scenes based either on
   hard-coded preferences or user choice.  Once this choice has been
   made, the consumer would then decide how to configure the
   provider's encoding groups in order to make best use of the
   available network bandwidth and its own decoding capabilities.

11.4.1. One screen consumer

   VC3, VC4 and VC5 are all different entries by themselves, not
   grouped together in a single entry, so the receiving device should
   choose between one of those.  The choice would come down to
   whether to see the greatest number of participants simultaneously
   at roughly equal precedence (VC5), a switched view of just the
   loudest region (VC3) or a switched view with PiPs (VC4).  An
   endpoint device with a small amount of knowledge of these
   differences could offer a dynamic choice of these options, in-
   call, to the user.





Duckworth et. al.       Expires June 24, 2013               [Page 37]


Internet-Draft       CLUE Telepresence Framework        December 2012


11.4.2. Two screen consumer configuring the example

   Mixing systems with an even number of screens, "2n", and those
   with "2n+1" cameras (and vice versa) is always likely to be the
   problematic case.  In this instance, the behavior is likely to be
   determined by whether a "2 screen" system is really a "2 decoder"
   system, i.e., whether only one received stream can be displayed
   per screen or whether more than 2 streams can be received and
   spread across the available screen area.  To enumerate 3 possible
   behaviors here for the 2 screen system when it learns that the far
   end is "ideally" expressed via 3 capture streams:

   1. Fall back to receiving just a single stream (VC3, VC4 or VC5 as
      per the 1 screen consumer case above) and either leave one
      screen blank or use it for presentation if / when a
      presentation becomes active.

   2. Receive 3 streams (VC0, VC1 and VC2) and display across 2
      screens (either with each capture being scaled to 2/3 of a
      screen and the centre capture being split across 2 screens) or,
      as would be necessary if there were large bezels on the
      screens, with each stream being scaled to 1/2 the screen width
      and height and there being a 4th "blank" panel.  This 4th panel
      could potentially be used for any presentation that became
      active during the call.

   3. Receive 3 streams, decode all 3, and use control information
      indicating which was the most active to switch between showing
      the left and centre streams (one per screen) and the centre and
      right streams.

   For an endpoint capable of all 3 methods of working described
   above, again it might be appropriate to offer the user the choice
   of display mode.

11.4.3. Three screen consumer configuring the example

   This is the most straightforward case - the consumer would look to
   identify a set of streams to receive that best matched its
   available screens and so the VC0 plus VC1 plus VC2 should match
   optimally.  The spatial ordering would give sufficient information
   for the correct video capture to be shown on the correct screen,
   and the consumer would either need to divide a single encoding
   group's capability by 3 to determine what resolution and frame
   rate to configure the provider with or to configure the individual


Duckworth et. al.       Expires June 24, 2013               [Page 38]


Internet-Draft       CLUE Telepresence Framework        December 2012


   video captures' encoding groups with what makes most sense (taking
   into account the receive side decode capabilities, overall call
   bandwidth, the resolution of the screens plus any user preferences
   such as motion vs sharpness).

12. Acknowledgements

   Mark Gorzyinski contributed much to the approach.  We want to
   thank Stephen Botzko for helpful discussions on audio.

13. IANA Considerations

   TBD

14. Security Considerations

   TBD

15. Changes Since Last Version

   NOTE TO THE RFC-Editor: Please remove this section prior to
   publication as an RFC.

   Changes from 06 to 07:

   1. Ticket #9.  Rename Axis of Capture Point attribute to Point on
      Line of Capture.  Clarify the description of this attribute.

   2. Ticket #17.  Add "capture encoding" definition.  Use this new
      term throughout document as appropriate, replacing some usage
      of the terms "stream" and "encoding".

   3. Ticket #18.  Add Max Capture Encodings media capture attribute.

   4. Add clarification that different capture scene entries are not
      necessarily mutually exclusive.

   Changes from 05 to 06:

   1. Capture scene description attribute is a list of text strings,
      each in a different language, rather than just a single string.

   2. Add new Axis of Capture Point attribute.

   3. Remove appendices A.1 through A.6.


Duckworth et. al.       Expires June 24, 2013               [Page 39]


Internet-Draft       CLUE Telepresence Framework        December 2012


   4. Clarify that the provider must use the same coordinate system
      with same scale and origin for all coordinates within the same
      capture scene.

   Changes from 04 to 05:

   1. Clarify limitations of "composed" attribute.

   2. Add new section "capture scene entry attributes" and add the
      attribute "scene-switch-policy".

   3. Add capture scene description attribute and description
      language attribute.

   4. Editorial changes to examples section for consistency with the
      rest of the document.

   Changes from 03 to 04:

   1. Remove sentence from overview - "This constitutes a significant
      change ..."

   2. Clarify a consumer can choose a subset of captures from a
      capture scene entry or a simultaneous set (in section "capture
      scene" and "consumer's choice...").

   3. Reword first paragraph of Media Capture Attributes section.

   4. Clarify a stereo audio capture is different from two mono audio
      captures (description of audio channel format attribute).

   5. Clarify what it means when coordinate information is not
      specified for area of capture, point of capture, area of scene.

   6. Change the term "producer" to "provider" to be consistent (it
      was just in two places).

   7. Change name of "purpose" attribute to "content" and refer to
      RFC4796 for values.

   8. Clarify simultaneous sets are part of a provider advertisement,
      and apply across all capture scenes in the advertisement.

   9. Remove sentence about lip-sync between all media captures in a
      capture scene.


Duckworth et. al.       Expires June 24, 2013               [Page 40]


Internet-Draft       CLUE Telepresence Framework        December 2012


   10.   Combine the concepts of "capture scene" and "capture set"
      into a single concept, using the term "capture scene" to
      replace the previous term "capture set", and eliminating the
      original separate capture scene concept.

   Informative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC3261]  Rosenberg, J., Schulzrinne, H., Camarillo, G.,
   Johnston,
              A., Peterson, J., Sparks, R., Handley, M., and E.
              Schooler, "SIP: Session Initiation Protocol", RFC 3261,
              June 2002.

   [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
              Jacobson, "RTP: A Transport Protocol for Real-Time
              Applications", STD 64, RFC 3550, July 2003.

   [RFC4353]  Rosenberg, J., "A Framework for Conferencing with the
              Session Initiation Protocol (SIP)", RFC 4353,
              February 2006.

   [RFC4796]  Hautakorpi, J. and G. Camarillo, "The Session
   Description
              Protocol (SDP) Content Attribute", RFC 4796,
              February 2007.

   [RFC5117]  Westerlund, M. and S. Wenger, "RTP Topologies", RFC
   5117,
              January 2008.

   [RFC5646]  Phillips, A. and M. Davis, "Tags for Identifying
              Languages", BCP 47, RFC 5646, September 2009.

   [IANA-Lan]
              IANA, "Language Subtag Registry",
   <http://www.iana.org/assignments/
              language-subtag-registry>.







Duckworth et. al.       Expires June 24, 2013               [Page 41]


Internet-Draft       CLUE Telepresence Framework        December 2012


16. Authors' Addresses

   Mark Duckworth (editor)
   Polycom
   Andover, MA  01810
   USA

   Email: mark.duckworth@polycom.com



   Andrew Pepperell
   Silverflare
   Uxbridge, England
   UK

   Email: apeppere@gmail.com



   Stephan Wenger
   Vidyo, Inc.
   433 Hakcensack Ave.
   Hackensack, N.J. 07601
   USA

   Email: stewe@stewe.org




















Duckworth et. al.       Expires June 24, 2013               [Page 42]


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/