[Docs] [txt|pdf] [Tracker] [Email] [Nits]

Versions: 00

CLUE                                                          A. Romanow
Internet-Draft                                                 R. Hansen
Intended status: Standards Track                           Cisco Systems
Expires: December 2, 2012                                   A. Pepperell
                                                             Silverflare
                                                              B. Baldino
                                                           Cisco Systems
                                                            May 31, 2012


    The need for audio rendering tag mechanism in the CLUE Framework
               draft-romanow-clue-audio-rendering-tag-00

Abstract

   The purpose of this draft is for discussion in the CLUE working
   group.

   It proposes adding an audio rendering tag to the CLUE framework
   [I-D.ietf-clue-framework], which makes it possible for the consumer
   to correctly render audio with respect to video in a multistream
   video conference.  The solution proposed is in partial response to
   CLUE Task #10, Does Framework provide sufficient info for receiver?

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on December 2, 2012.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents



Romanow, et al.         Expires December 2, 2012                [Page 1]


Internet-Draft        Audio rendering tag for CLUE              May 2012


   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.


Table of Contents

   1.  Motivation- the issue . . . . . . . . . . . . . . . . . . . . . 3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 3
   3.  Audio Rendering Tag Mechanism . . . . . . . . . . . . . . . . . 3
   4.  Use of the RTP header extension . . . . . . . . . . . . . . . . 5
   5.  Use case note . . . . . . . . . . . . . . . . . . . . . . . . . 6
   6.  Security Considerations . . . . . . . . . . . . . . . . . . . . 6
   7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . 6
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6
   9.  References  . . . . . . . . . . . . . . . . . . . . . . . . . . 7
     9.1.  Normative References  . . . . . . . . . . . . . . . . . . . 7
     9.2.  Informative References  . . . . . . . . . . . . . . . . . . 7
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . . . 7




























Romanow, et al.         Expires December 2, 2012                [Page 2]


Internet-Draft        Audio rendering tag for CLUE              May 2012


1.  Motivation- the issue

   A goal for CLUE audio is that listeners perceive the direction of a
   sound source to be the same as that of the visual image of the
   source; this is referred to as directional audio.  In some situations
   the existing clue mechanisms are adequate.  The consumer can use the
   spatial information to correctly place the audio when the provider
   advertisement includes spatial information (point of origin and
   capture area) giving a static relationship between both video and
   associated audio captures.

   However, in some circumstances, for different reasons, the audio
   and/or video spatial information is not sent in the provider
   advertisement.  For instance, the case of a three-screen system
   advertising three video captures and one switched audio capture,
   where the audio is switched from the loudest of three microphones.
   In this case, how will the consumer know how to associate the audio
   with the correct video so it can be played out in the correct
   location?

   Here we suggest a simple mechanism -- audio rendering tagging.

   When audio and video cannot be matched through provider advertisement
   spatial information, we would like the ability to play out audio on
   multiple speakers matching the position of the speaker in the
   original scene.  Also, the audio may be assigned to a speaker in
   real-time.  It may need to be mixe locally and played out on any
   speaker.  For example, if the consumer wants to hear the top 3
   speakers, regardless of where they are located remotely, if all 3 top
   speakers are coming from the left, then the 3 speakers need to be
   mixed, perhaps locally, and played out on the left.

   Note: Several typical scenarios are described at the end of this note
   in section titled Use Case.


2.  Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119] and
   indicate requirement levels for compliant implementations.


3.  Audio Rendering Tag Mechanism

   We propose an audio tagging mechanism In order to cope with a
   changing mapping of the most significant audio and video participants



Romanow, et al.         Expires December 2, 2012                [Page 3]


Internet-Draft        Audio rendering tag for CLUE              May 2012


   (i.e., normal MCU operations in the presence of more participants'
   media streams that can be rendered simultaneously) and to get audio
   played out correctly to multiple speakers.  A consumer optionally
   tells the provider an audio tag value corresponding to each of its
   chosen video captures which enables received audio to be associated
   with the correct video stream, even when the set of audible
   participants changes.  This information is included with the consumer
   request so there is no need for additional CLUE message exchanges
   (specifically, no additional provider capture advertisements or
   consumer requests).

   The audio tags are defined in the consumer request as opposed to in a
   capture advertised by a producer.  The reason for this is that it is
   valid for a consumer to request a capture multiple times (with
   different encodings, for example) and hence a method is required for
   differentiating between these streams.

   When the consumer configures the provider, saying which captures it
   wants, it also optionally includes an audio tag with each capture
   request.  For example, VC1, ATag1; VC2, ATag2.  When the provider
   sends audio packets to the consumer, it includes the appropriate
   audio tag in an RTP header extension.  For example, if the provider
   is sending audio packets that are associated with VC1, it tags the
   packets with ATag1.  The consumer can then play out the audio in a
   position appropriate for video from VC1.

   Suppose that several audio streams need to be played out through the
   same speaker - for example, the 3 audio streams (AC1, AC2, AC3) need
   to be played out at the speaker associated with VC1.  The provider
   would send:


   AC1  ATag1
   AC2  ATag1
   AC3  ATag1


   AC1, AC2 and AC3 are all played out on the same speaker, the audio
   output associated with VC1.  This takes care of the issue of dynamic
   audio output - assigning the right speaker to audio streams.

   Figure 1 illustrates an example showing 3 screens, each with a main
   video and 3 PIPs.  Below each screen is a list of the video captures,
   VCs with the associated Audio Tag.







Romanow, et al.         Expires December 2, 2012                [Page 4]


Internet-Draft        Audio rendering tag for CLUE              May 2012


          ----------------------3 Screens ---------------------
         |------------------+- -----------------+------------------Y
         |                  |                   |                  |
         |    VC1           |    VC2            |    VC3           |
         |                  |                   |                  |
         |                  |                   |                  |
         |                  |                   |                  |
         | ''''|'''''''''|  |  ''''|'''''|'''|  |  '''''|''''|''''||
         | |VC4|.VC5.|VC6|  |  |VC7|.VC8.|VC9|  |  |VC10|VC11|VC12||
         '------------------+-------------------+-------------------
           VC1                  VC2                 VC3
           VC4  Audio Tag 1     VC7  Audio tag 2    VC10 Audio tag 3
           VC5                  VC8                 VC11
           VC6                  VC9                 VC12


            Figure 1: Audio rendering tags for 3 screen example

   The provider may choose not to include the extension header in an
   audio packet, signaling that there is no association between the
   current audio and current video (i.e., an audio-only participant).
   It may also include more than one audio tag in the extension header,
   signaling that this audio is associated with multiple current video
   participants, due perhaps to a capture being received multiple times
   at different resolutions, or two video captures that both include the
   current speaker.

   This mechanism also allows multiple audio streams to be associated
   with a single video stream (i.e. for a composed video stream); this
   simply requires the appropriate audio packets to be tagged with the
   same id.


4.  Use of the RTP header extension

   We propose that audio tags are integer numbers between 0 and 255
   optionally set by the consumer per requested capture.  This allows up
   to 16 tags to be included in a one-byte RTP header extension [RFC
   5285].  An example header extension for an audio packet with one tag
   follows.  The audio tag extension is ID1.  The example includes
   another header extension (ID0) to show how the proposal would
   interact with [I-D.lennox-clue-rtp-usage]:


   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |       0xBE    |    0xDE       |           length=1            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+



Romanow, et al.         Expires December 2, 2012                [Page 5]


Internet-Draft        Audio rendering tag for CLUE              May 2012


   |  ID0  | L=0  |     data      |  ID1  |  L=0  |     Tag       |
   -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


          RTP ext headers for audio rendering tag and capture ID

   The lack of the RTP header extension in a packet means that the audio
   packet is not associated with any of the requested video streams that
   included audio tags.


5.  Use case note

   o  An endpoint can receive multiple video and audio streams and
      render complex layouts locally.
   o  It may have a wide display area so directional audio is important.
   o  It may have one loudspeaker per display, or perhaps some entirely
      different multi-loudspeaker setup known only to the endpoint
      itself.
   o  The endpoint may therefore have the capability of playing back
      audio from a wide range of positions.
   o  Either from a few fixed zones or with fine granularity.
   o  Either by routing a sound source to a single loudspeaker, by
      panning between pairs of loudspeakers, or by some other advanced
      distribution scheme involving several or even all loudspeakers.


6.  Security Considerations

   TBD


7.  Acknowledgements

   Thanks to Johan Nielsen for discussions and adding the Use case
   note.cuss


8.  IANA Considerations

   TBD


9.  References







Romanow, et al.         Expires December 2, 2012                [Page 6]


Internet-Draft        Audio rendering tag for CLUE              May 2012


9.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

9.2.  Informative References

   [I-D.ietf-clue-framework]
              Romanow, A., Duckworth, M., Pepperell, A., and B. Baldino,
              "Framework for Telepresence Multi-Streams",
              draft-ietf-clue-framework-05 (work in progress), May 2012.

   [I-D.lennox-clue-rtp-usage]
              Lennox, J., Witty, P., and A. Romanow, "Real-Time
              Transport Protocol (RTP) Usage for Telepresence Sessions",
              draft-lennox-clue-rtp-usage-03 (work in progress),
              March 2012.


Authors' Addresses

   Allyn Romanow
   Cisco Systems
   San Jose, CA  95134
   USA

   Email: allyn@cisco.com


   Robert Hansen
   Cisco Systems
   Langley,
   UK

   Email: rohanse2@cisco.com


   Andy Pepperell
   Silverflare

   Email: andy.pepperell@silverflare.com










Romanow, et al.         Expires December 2, 2012                [Page 7]


Internet-Draft        Audio rendering tag for CLUE              May 2012


   Brian Baldino
   Cisco Systems
   San Jose, CA  95134
   USA

   Email: bbaldino@cisco.com













































Romanow, et al.         Expires December 2, 2012                [Page 8]


Html markup produced by rfcmarkup 1.124, available from https://tools.ietf.org/tools/rfcmarkup/