[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02

Internet Engineering Task Force                         G. Bertrand, Ed.
Internet-Draft                                                E. Stephan
Intended status: Informational                   France Telecom - Orange
Expires: February 11, 2013                                R. Peterkofsky
                                                           Skytide, Inc.
                                                         August 10, 2012


                         CDNI Logging Interface
                     draft-bertrand-cdni-logging-01

Abstract

   This memo specifies the Logging interface between a downstream CDN
   (dCDN) and an upstream CDN (uCDN).  It introduces a framework, an
   architecture design and a set of new requirements.  Then it drafts an
   information model.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on February 11, 2013.

Copyright Notice

   Copyright (c) 2012 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as



Bertrand, et al.        Expires February 11, 2013               [Page 1]


Internet-Draft                CDNI Logging                   August 2012


   described in the Simplified BSD License.

   This document may contain material from IETF Documents or IETF
   Contributions published or made publicly available before November
   10, 2008.  The person(s) controlling the copyright in some of this
   material may not have granted the IETF Trust the right to allow
   modifications of such material outside the IETF Standards Process.
   Without obtaining an adequate license from the person(s) controlling
   the copyright in such materials, this document may not be modified
   outside the IETF Standards Process, and derivative works of it may
   not be created outside the IETF Standards Process, except to format
   it for publication as an RFC or to translate it into languages other
   than English.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1.  Terminology  . . . . . . . . . . . . . . . . . . . . . . .  4
     1.2.  Abbreviations  . . . . . . . . . . . . . . . . . . . . . .  5
   2.  Logging Framework and Architecture . . . . . . . . . . . . . .  6
   3.  Additional Requirements  . . . . . . . . . . . . . . . . . . . 10
   4.  Rationale for Logging Interface  . . . . . . . . . . . . . . . 10
     4.1.  Usages of CDNI Logging Information . . . . . . . . . . . . 10
       4.1.1.  Maintenance/Debugging  . . . . . . . . . . . . . . . . 10
       4.1.2.  Accounting . . . . . . . . . . . . . . . . . . . . . . 11
       4.1.3.  End-User Experience Management . . . . . . . . . . . . 11
       4.1.4.  Security . . . . . . . . . . . . . . . . . . . . . . . 11
       4.1.5.  Legal Logging Duties . . . . . . . . . . . . . . . . . 11
     4.2.  Logging Information Views  . . . . . . . . . . . . . . . . 11
     4.3.  Information Extracted From Logging Data  . . . . . . . . . 12
   5.  Log Information Elements . . . . . . . . . . . . . . . . . . . 13
     5.1.  Information Elements . . . . . . . . . . . . . . . . . . . 13
     5.2.  Logging Record Information Elements for Content
           Delivery . . . . . . . . . . . . . . . . . . . . . . . . . 16
     5.3.  Logging Record Information Elements for  . . . . . . . . . 17
     5.4.  Logging Record Information Elements for Other
           Operations . . . . . . . . . . . . . . . . . . . . . . . . 17
   6.  Core Logging Records . . . . . . . . . . . . . . . . . . . . . 18
     6.1.  Content Delivery . . . . . . . . . . . . . . . . . . . . . 18
     6.2.  Content Acquisition  . . . . . . . . . . . . . . . . . . . 18
       6.2.1.  Logging Records Provided by dCDN to uCDN . . . . . . . 18
       6.2.2.  Logging Records Provided by uCDN to dCDN . . . . . . . 19
     6.3.  Content Invalidation and Purging . . . . . . . . . . . . . 19
     6.4.  Logging Extensibility  . . . . . . . . . . . . . . . . . . 20
   7.  Default Logging Information Format . . . . . . . . . . . . . . 20
     7.1.  Logging Files  . . . . . . . . . . . . . . . . . . . . . . 20
     7.2.  File Format  . . . . . . . . . . . . . . . . . . . . . . . 20



Bertrand, et al.        Expires February 11, 2013               [Page 2]


Internet-Draft                CDNI Logging                   August 2012


       7.2.1.  Headers  . . . . . . . . . . . . . . . . . . . . . . . 21
       7.2.2.  Body (Logging Records) Format  . . . . . . . . . . . . 21
       7.2.3.  Footer Format  . . . . . . . . . . . . . . . . . . . . 22
   8.  Logging Format and Scope Negotiation . . . . . . . . . . . . . 22
   9.  Logging Information Transport  . . . . . . . . . . . . . . . . 22
     9.1.  Major Requirements on Logging Protocols  . . . . . . . . . 23
     9.2.  Recommended Logging Protocol for Non Real-Time Logging . . 23
     9.3.  Recommended Logging Protocol for Real-Time Logging . . . . 24
   10. Logging Process  . . . . . . . . . . . . . . . . . . . . . . . 24
     10.1. Logging Aggregation  . . . . . . . . . . . . . . . . . . . 24
     10.2. Logging Filtering  . . . . . . . . . . . . . . . . . . . . 25
     10.3. Logging Update and Rectification . . . . . . . . . . . . . 26
   11. Open Issues  . . . . . . . . . . . . . . . . . . . . . . . . . 26
   12. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 27
   13. Security Considerations  . . . . . . . . . . . . . . . . . . . 27
     13.1. Privacy  . . . . . . . . . . . . . . . . . . . . . . . . . 27
     13.2. Non Repudiation  . . . . . . . . . . . . . . . . . . . . . 27
   14. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 27
   15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 28
     15.1. Normative References . . . . . . . . . . . . . . . . . . . 28
     15.2. Informative References . . . . . . . . . . . . . . . . . . 28
   Appendix A.  Examples Log Format . . . . . . . . . . . . . . . . . 29
     A.1.  W3C Common Log File (CLF) Format . . . . . . . . . . . . . 29
     A.2.  W3C Extended Log File (ELF) Format . . . . . . . . . . . . 30
     A.3.  National Center for Supercomputing Applications (NCSA)
           Common Log Format  . . . . . . . . . . . . . . . . . . . . 32
     A.4.  NCSA Combined Log Format . . . . . . . . . . . . . . . . . 32
     A.5.  NCSA Separate Log Format . . . . . . . . . . . . . . . . . 32
     A.6.  Squid 2.0 Native Log Format for Access Logs  . . . . . . . 32
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 33





















Bertrand, et al.        Expires February 11, 2013               [Page 3]


Internet-Draft                CDNI Logging                   August 2012


1.  Introduction

   This memo specifies the Logging interface between a downstream CDN
   (dCDN) and an upstream CDN (uCDN).  It introduces a framework, an
   architecture design and a set of new requirements.  Then it drafts an
   information model.

   The reader should be familiar with the work of the CDNI WG:

   o  CDNI problem statement [I-D.ietf-cdni-problem-statement] and
      framework [I-D.ietf-cdni-framework] identify a Logging interface,

   o  Section 7 of [I-D.ietf-cdni-requirements] specifies a set of
      requirements for Logging,

   o  [I-D.ietf-cdni-use-cases] outlines real world use-cases for
      interconnecting CDNs.  These use cases require the exchange of
      Logging information between the dCDN and the uCDN.

   o  [I-D.lefaucheur-cdni-logging-delivery] complements the present
      memo by proposing CDNI Logging formats for content deliveries
      performed using HTTP or HTTP adaptive streaming.

   The present document describes:

   o  The Logging framework and architecture (Section 2),

   o  The requirements (Section 3),

   o  Discussion on the monitoring and the reporting use cases
      (Section 4)

   o  Log information (Section 5 and Section 6),

1.1.  Terminology

   In this document, the first letter of each CDNI-specific term is
   capitalized.  We adopt the terminology described in
   [I-D.ietf-cdni-problem-statement] and [I-D.ietf-cdni-framework], and
   extend it with the additional terms defined below.

   For clarity, we use the word "Log" only for referring to internal CDN
   logs and we use the word "Logging" for any inter-CDN information
   exchange and processing operations related to CDNI Logging interface.
   Log and Logging formats may be different.

   Log: CDN internal information collection and processing operations.




Bertrand, et al.        Expires February 11, 2013               [Page 4]


Internet-Draft                CDNI Logging                   August 2012


   Logging: Inter-CDN information exchange and processing operations.

   Fragmented object: [Ed.  Note: Tentative of a simple definition which
   fits with the current CDNI charter] Fragmented objects are pieces of
   content provided by a CSP which are delivered individually through a
   CDN interconnection.  They differ from a simple object because the
   delivery of the content to one user agent may be provided by more
   than one Surrogate/CDN.

   CDN Reporting: the process of providing the relevant information that
   will be used to create a formatted content delivery report provided
   to the CSP in differed time.  Such information typically includes
   aggregated data that can cover a large period of time (e.g., from
   hours to several months).  One of the usages of reporting is the
   collection of charging data related to CDN services and the
   computation of Key Performance Indicators (KPIs).

   CDN Monitoring: the process of providing content delivery information
   in real-time.  The monitoring typically includes data in real time to
   provide a vision of the deliveries in progress, for service operation
   purposes.  It presents a view of the global health of the services as
   well as information on usage and performance, for network services
   supervision and operation management.  In particular, monitoring data
   can be used to generate alarms.

   End-User experience management: study of Logging data using
   statistical analysis to discover, understand, and predict user
   behavior patterns.

   Delivery Service: a specific instantiation of content delivery
   service configuration.  For instance, a given uCDN (uCDN1) may
   request a given dCDN (dCDN1) to configure a Delivery Service for
   handling requests for HTTP Adaptive streaming videos delegated by
   uCDN1 and related to a specific CSP (CSP1), and another one for
   handling request for static pictures delegated by uCDN1 and related
   to CSP1.  In this simple example, uCDN1 may request dCDN1 to include
   delivery service information in its CDNI Logging, to help uCDN
   provide relevant reports to CSP1.

1.2.  Abbreviations

   o  API: Application Programming Interface

   o  CCID: Content Collection Identifier

   o  CDN: Content Delivery Network





Bertrand, et al.        Expires February 11, 2013               [Page 5]


Internet-Draft                CDNI Logging                   August 2012


   o  CDNP: Content Delivery Network Provider

   o  CoDR: Content Delivery Record

   o  CSP: Content Service Provider

   o  DASH: Dynamic Adaptive Streaming over HTTP

   o  dCDN: downstream CDN

   o  FTP: File Transfer Protocol

   o  HAS: HTTP Adaptive Streaming

   o  KPI: Key Performance Indicator

   o  PVR: Personal Video Recorder

   o  SID: Session Identifier

   o  SFTP: SSH File Transfer Protocol

   o  SNMP: Simple Network Management Protocol

   o  uCDN: upstream CDN


2.  Logging Framework and Architecture

   The framework of the Logging interface is straightforward: dCDN logs
   any information related to the completion of any task performed by a
   dCDN on behalf of an uCDN and any exchange related to the management
   of the contents that the said dCDN delivers on behalf of an uCDN, as
   discussed in Section 6.1.

   Logging is a mandatory feature for a CDN, especially if the CDN is
   interconnected to other CDNs.  Logging provides the raw material for
   some essential operations of a delivery service, such as monitoring,
   reporting, billing, etc.

   As stated in [I-D.ietf-cdni-problem-statement], "the CDNI Logging
   interface enables details of logs or events to be exchanged between
   interconnected CDNs".

   Figure 1 provides an example of Logging information exchanges. uCDN
   is connected to dCDN-1 and dCDN-2.  Both dCDN-1, dCDN-2, and uCDN
   deliver content for CSP.  The Logging interface enables the uCDN to
   obtain Logging data from dCDN-1 and dCDN-2.  In the example, uCDN



Bertrand, et al.        Expires February 11, 2013               [Page 6]


Internet-Draft                CDNI Logging                   August 2012


   uses the Logging data:

   o  to analyze the performance of the delivery operated by the dCDNs
      and to adjust its operations (e.g., request routing) as
      appropriate,

   o  to provide reporting (non real-time) and monitoring (real-time)
      information to CSP.

   For instance, uCDN merges Logging data, extracts relevant KPIs, and
   presents a formatted report to CSP, in addition to a bill for the
   content delivered. uCDN may also provide Logging data as raw log
   files to CSP, so that CSP uses its own Logging analysis tools.


                   +-----+
                   | CSP |
                   +-----+
                      ^ Reporting and monitoring data
                      | Billing
                   ,--,--.
       Logging  ,-'       `-.   Logging
       Data    (     uCDN    )  Data
          ....> `-.       _,-'<....
          |        `-'-'-'        |
       ,--v--.       ^ ^       ,--v--.
    ,-'       `-.    | |    ,-'       `-.
   (   dCDN-1    )<--+ +-->(   dCDN-2    )  Logging
    `-.       ,-'  Logging  `-.      _,-'<...Data
      `--'--'       Tuning     `--'-'        |
                                  ^       ,--|--.
                          Logging |     ,'       `-.
                           Tuning + -->(  dCDN-3    )
                                        `.       ,-'
                                          `--'--'


                 Figure 1: Exchange of Logging Information

   A dCDN integrates the logging of its downstream CDNs in the Logging
   that it provides to the uCDN, as required by
   [I-D.ietf-cdni-requirements] (LOG-3).

   Figure 1 represents bi-directional arrows between dCDN and uCDN for
   the exchange of Logging data, because even if the common case
   involves the uCDN retrieving Logging data on the dCDN, the reverse
   case where the dCDN retrieves Logging data (e.g., related to dCDN's
   content acquisition requests to the uCDN) on the uCDN is also



Bertrand, et al.        Expires February 11, 2013               [Page 7]


Internet-Draft                CDNI Logging                   August 2012


   possible.

   Note that the format of Logging data that the dCDN provides might be
   different from the one that the dCDN uses internally.  In this case,
   the dCDN needs to reformat the Logging data before it provides this
   data to the uCDN.  Similarly, an uCDN might reformat the Logging data
   that it receives before providing it to the CSP or to its uCDN.  Such
   reformatting operations are time consuming (delays in the Logging
   chain) and introduce a processing burden.  Therefore, it is
   recommended that the CDNI Logging format be as close as possible from
   the most common CDN Log formats.

   Figure 2 presents the Logging Architecture.  More details on the
   Logging operations are provided in Section 10.  A dCDN prepares the
   Logging data requested by the uCDN.  This preparation involves
   operations such as filtering, aggregating, anonymizing, and
   summarizing the logs.  The uCDN downloads the corresponding Logging
   Records and performs its own reporting for the CSP.

































Bertrand, et al.        Expires February 11, 2013               [Page 8]


Internet-Draft                CDNI Logging                   August 2012


  +------+
  | CSP  |
  +------+
    ^
    ^ Reporting, Monitoring, Billing
    ^
 ---^---------------------   Logging Record   -------------------------
/   ^    Upstream CDN     \    selection     /    Downstream CDN       \
|+-----+  +-------------+ | and format nego. | +-------------+  +-----+|
||     |**|   Control   | |<---------------->| |   Control   |**|     ||
||     |  +-------------+ |                  | +-------------+  | I   ||
|| I   |                  |                  |                  | n   ||
|| n   |  +-------------+ |                  | +-------------+  | t   ||
|| t   |<<|   Logging   | |                  | |   Logging   |<<| e   ||
|| e   |  +-------------+ |<---------------->| +-------------+  | r   ||
|| r   |                  | Logging Records  |                  | c L ||
|| c L |                  |                  |                  | o o ||
|| o o |  +-------------+ |                  | +-------------+  | n g ||
|| n g |<<|Req-Routing  | |                  | |Req-Routing  |>>| n i ||
|| n i |  +-------------+ |                  | +-------------+  | e c ||
|| e c |                  |                  |                  | c   ||
|| c   |  +-------------+ |                  | +-------------+  | t   ||
|| t   |<<| Metadata    | |                  | | Metadata    |>>| i   ||
|| i   |  +-------------+ |                  | +-------------+  | o   ||
|| o   |                  |                  |                  | n   ||
|| n   |  +-------------+ |                  | +-------------+  |     ||
||     |<<| Distribution| |******************| | Distribution|>>|     ||
|+-----+  +-------------+ |   Acquisition    | +-------------+  +-----+|
\                         /                  \         . *             /
 -------------------------                    ---------.-*-------------
               .                                       . *
               .                               Request . * Delivery
               .                                    +--.-*--+
               ..................Request............| User  |
                                                    | Agent |
                                                    +-------+


                      Figure 2: Logging Architecture

   In Figure 2, the Logging Record selection and format negotiation
   occurs at Control Interface level, as these operations provide static
   information for initializing the Logging interface.

   Logging data captures information elements that may be available at
   various stages during the life-cycle of content distribution.  The
   arrows (">>") in Figure 2 represent the direction of information
   elements in the Logging process.



Bertrand, et al.        Expires February 11, 2013               [Page 9]


Internet-Draft                CDNI Logging                   August 2012


3.  Additional Requirements

   Section 7 of [I-D.ietf-cdni-requirements], already specifies a set of
   requirements for Logging (LOG-1 to LOG-16).  Some security
   requirements also affect Logging (e.g., SEC-4).


4.  Rationale for Logging Interface

   [I-D.ietf-cdni-framework] and [I-D.ietf-cdni-problem-statement]
   introduce the rationale for the Logging interface as a means for an
   uCDN to acquire some visibility on the contents the dCDN delivers on
   behalf of the uCDN. dCDN provides the uCDN with elements of
   information and Logging Records for operating the CDN interconnection
   and reporting to the CSP.  This section develops use cases that
   require exchange of Logging information.

4.1.  Usages of CDNI Logging Information

   This section presents the usage of the Logging Records by an uCDN.
   It does not make any assumption on where the Logging Records are
   produced.  Logging Records may be produced either by the uCDN or a
   dCDN.

4.1.1.  Maintenance/Debugging

   Logging is useful to permit the detection (and limit the risk) of
   content delivery failures.  In particular, Logging facilitates the
   resolution of false configuration issues.

   To detect faults, Logging must enable the reporting of any CDN
   operation success and failure, such as request redirection, content
   acquisition, etc.  The uCDN can summarize such information into KPIs.
   For instance, Logging format should allow the computation of the
   number of times during a given epoch, a content delivery related to a
   specific service succeeds/fails.

   Logging is useful to analyze the performance of content delivery
   services.  This implies computing KPIs from the Logging data for
   service quality analysis and monitoring (see Section 4.3).

   Logging enables the CDN providers to evaluate the QoS level related
   to a specific delivery service.  For instance, one aspect of this QoS
   level could be measured through the average delivery throughput
   experienced by End-Users in a given region for this specific service
   over a period of time.

   Logging enables the CDN providers to identify and troubleshoot



Bertrand, et al.        Expires February 11, 2013              [Page 10]


Internet-Draft                CDNI Logging                   August 2012


   performance degradations.  In particular, Logging enables the
   communication of traffic data (e.g., the amount of traffic that has
   been forwarded by a dCDN on behalf of an uCDN over a given period of
   time), which is particularly useful for CDN and network planning
   operations.

4.1.2.  Accounting

   Logging is essential for accounting, to permit inter-CDN billing, and
   CSP billing by uCDN.  For instance, Logging enables the uCDN to check
   the total amount of traffic delivered by every dCDN and for every
   delivery service, as well as, the associated bandwidth usage (e.g.,
   peak, 95th percentile), and the maximum number of simultaneous
   sessions over a given period of time.

4.1.3.  End-User Experience Management

   The goal of End-User experience management is to gather any relevant
   information to meter audience, analyze user behavior, etc.  For
   instance, Logging enables the CDN providers to report on content
   consumption (e.g., delivered sessions per content) in a specific
   geographic area.

4.1.4.  Security

   The goal of security is to prevent and monitor unauthorized access,
   misuse, modification, and denial of access of a service.  A set of
   information is logged for security purposes.  In particular, access
   to content is usually collected to permit the CSP to detect
   infringements of content delivery policies and other abnormal End-
   User behaviors.

4.1.5.  Legal Logging Duties

   Depending on the country considered, the CDNs may have to retain
   specific Logging information during a legal retention period, to
   comply with judicial requisitions.

4.2.  Logging Information Views

   Logging information is useful to the uCDN and potentially to the CSP.
   Different views of the Logging information may be provided depending
   on privacy, business, and scalability constraints.  Some kind of
   information format adaptation capability may be supported by an uCDN
   to present some (e.g., filtered, aggregated) data in the appropriate
   format (raw log files, reports) to the CSP.  More details on these
   operations are provided in Section 10.




Bertrand, et al.        Expires February 11, 2013              [Page 11]


Internet-Draft                CDNI Logging                   August 2012


   We provide a non-exhaustive list and description of tools that can be
   fed with Logging information.

   o  Tools used by the uCDN's operator: billing tools (information
      system), customer experience intelligence, reporting tools,
      security auditing tools, dimensioning tools, strategic planning
      and investment...

   o  Tools used by CSPs: customer experience management tools,
      reporting tools, security auditing tools...

4.3.  Information Extracted From Logging Data

   This section presents, for explanatory purposes, a non-exhaustive
   list of information that can be extracted/produced from logs.
   Depending on the inter-CDN agreement, this information may be
   computed by the uCDN or by the dCDN.  Nevertheless, it is usually the
   uCDN that computes KPIs, because uCDN and dCDN may have different
   definitions of the KPIs and the computation of some KPIs requires a
   vision of all the deliveries performed by the uCDN and all its dCDNs.

   CSPs require specific information, such as KPIs, about the delivery
   of their content.  The Logging data must contain appropriate
   information to enable CSPs or the uCDN to extract the required KPIs.
   In the present section, we list important examples of KPIs:

   o  Number of delivery requests received from End-Users in a given
      region for each piece of content, during a given period of time
      (e.g., hour/day/week/month),

   o  Percentage of delivery successes / failures among the
      aforementioned requests

   o  Number of failures listed by failure type (e.g., HTTP error code)
      for requests received from End-Users in a given region and for
      each piece of content, during a given period of time (e.g., hour/
      day/week/month),

   o  Number and cause of delivery premature termination for End-Users
      in a given region and for each piece of content, during a given
      period of time (e.g., hour/day/week/month),

   o  Maximum and mean number of simultaneous sessions established by
      End-Users in a given region, for a given delivery service, and
      during a given period of time (e.g., hour/day/week/month),

   o  Volume of traffic delivered for sessions established by End-Users
      in a given region, for a given delivery service, and during a



Bertrand, et al.        Expires February 11, 2013              [Page 12]


Internet-Draft                CDNI Logging                   August 2012


      given period of time (e.g., hour/day/week/month),

   o  Maximum, mean, and minimum delivery throughput for sessions
      established by End-Users in a given region, for a given delivery
      service, and during a given period of time (e.g., hour/day/week/
      month)

   o  Cache-hit and byte-hit ratios for requests received from End-Users
      in a given region for each piece of content, during a given period
      of time (e.g., hour/day/week/month)

   o  Top 10 of the most popular requested content (with time
      repartition into day/week/month),

   o  Terminal type (mobile, PC, STB, if this information can be
      acquired from the browser type header, for example).

   Additional KPIs can be computed from other sources of information
   than the Logging, for instance, data collected by a content portal or
   by specific client-side APIs.  Such KPIs are out of scope for the
   present memo.


5.  Log Information Elements

   CDNI must specify a set of Logging information elements to avoid log
   format regeneration, which would affect the performance of the log
   handling chain.  A common set of Logging information element eases
   the sharing of logs among the CDNs and the use of log processing
   tools, for instance, to prepare reporting.

   Existing CDNs Logging functions collect and consolidate logs
   performed by their Surrogates.  Surrogates usually store the logs
   using a format derived from Web servers' and caching proxies' log
   standards such as W3C, NCSA [ELF] [CLF], or Squid format [squid].  In
   practice, these formats are adapted to cope with CDN specifics.
   Appendix A presents examples of commonly used log formats.

5.1.  Information Elements

   This section describes a set of information elements that structure
   Logging information generated by the dCDN.  The section does not
   prescribe a particular encoding (such as SNMP SMI or alternatives).
   All fields in the Logging information are optional unless stated
   otherwise.  However, if a given CDN decides to support some of the
   Logging information fields, it must conform to the definition and
   format of this field specified in the present memo, to guarantee that
   interconnected CDNs share a common understanding of the Logging



Bertrand, et al.        Expires February 11, 2013              [Page 13]


Internet-Draft                CDNI Logging                   August 2012


   semantic and syntax.

   +-------------+-----------------------------------------------------+
   | Name        | Description                                         |
   +-------------+-----------------------------------------------------+
   | Start-time  | A start date and time associated with a logged      |
   |             | event; for instance, the time at which a Surrogate  |
   |             | received a content delivery request or the time at  |
   |             | which an origin server received a content           |
   |             | acquisition request.                                |
   | End-time    | An end date and time associated with a logged       |
   |             | event.  For instance, the time at which a Surrogate |
   |             | completed the handling of a content delivery        |
   |             | request (e.g., end of delivery or error).           |
   | Duration    | The duration of an operation in milliseconds.  For  |
   |             | instance, this field could be used to provide the   |
   |             | time it took to the Surrogate to send the requested |
   |             | file to the End-User, or the time it took the       |
   |             | Surrogate to acquire the file on a cache-miss       |
   |             | event.                                              |
   | Client-IP   | The IP address of the User Agent that issued the    |
   |             | logged request (or of a proxy).                     |
   | Operation   | The kind of operation that is logged; for instance, |
   |             | Acquisition, Delivery, or Purging.                  |
   | URI_full    | The full requested URL (e.g.,                       |
   |             | "http://node1.peer-a.op-b.net/cdn.csp.com/movies/po |
   |             | tter.avi?param=11&user=toto").  When HTTP request   |
   |             |  redirection is used, this URI includes the         |
   |             |  Surrogate FQDN.  If the association of requests to |
   |             |  Surrogates is confidential, the dCDN can present   |
   |             |  only URI_part to uCDN.                             |
   | URI_part    | The requested URL path (e.g.,                       |
   |             | /cdn.csp.com/movies/potter.avi?param=11&user=toto   |
   |             | if the full request URL was                         |
   |             | "http://node1.peer-a.op-b.net/cdn.csp.com/movies/po |
   |             | tter.avi?param=11&user=toto").  The URI without     |
   |             |  host-name typically includes the "CDN domain"      |
   |             |  (ex.cdn.csp.com) - cf. [I-D.ietf-cdni-framework]:  |
   |             |  it enables the identification of the CSP service   |
   |             |  agreed between the CSP and the CDNP operating the  |
   |             |  uCDN.                                              |
   | Protocol    | The protocol and protocol version of the message    |
   |             | that triggered the Logging entry.                   |
   | Request-met | The protocol method of the request message that     |
   | hod         | triggered the Logging entry.                        |
   | Status      | The protocol method of the reply message related to |
   |             | the Logging entry                                   |




Bertrand, et al.        Expires February 11, 2013              [Page 14]


Internet-Draft                CDNI Logging                   August 2012


   | Bytes-Trans | The number of bytes at application-layer            |
   | ferred      | protocol-level (e.g., HTTP) of the reply message    |
   |             | related to the Logging entry.  It includes the size |
   |             | of the response headers.                            |
   | Bytes-recei | The number of bytes (headers + body) of the message |
   | ved         | that triggered the Logging entry.                   |
   | Referrer    | The value of the Referrer header in an HTTP         |
   |             | request.                                            |
   | User-Agent  | The value of the User Agent header in an HTTP       |
   |             | request.                                            |
   | Cookie      | The value of the Cookie header in an HTTP request.  |
   | Record-dige | A digest of the Logging Record; it enables          |
   | st          | detecting corrupted Logging Records.                |
   | CCID        | A Content Collection IDentifier (CCID) eases the    |
   |             | correlation of several Logging Records related to a |
   |             | Content Collection (e.g., a movie split in chunks). |
   | SID         | A Session Identifier (SID) eases the correlation    |
   |             | (and aggregation) of several Logging Records        |
   |             | related to a session.  The SID is especially        |
   |             | relevant for summarizing HAS Logging information    |
   |             | [I-D.brandenburg-cdni-has].                         |
   +-------------+-----------------------------------------------------+

               Table 1: Logging Record Information Elements

   NB: we define three fields related to the timing of logged
   operations: Start-time, End-time, and Duration.  Only two of these
   three fields are required to obtain relevant timing information on
   the operation.  Start-time is typically useful for human readers
   (e.g., while debugging), however, most servers log the operations
   End-time which correspond to the time of log record generation.

   Multiple header fields, in addition to User Agent and Referrer, could
   be reproduced in the Logging entries.

   Note that uCDN may want to filter Logging data by user (and not by IP
   address) to provide more relevant information to the CSP.  In such
   case, a user may be identified as a combination of several pieces of
   information such as the client IP and User Agent or through the SID.

   The URI_full provides information on the Surrogate that provided the
   content.  This information can be relevant, for instance, for Inter-
   Affiliates scenarios [I-D.ietf-cdni-use-cases].  However, in some
   cases it may be considered as confidential and the dCDN may provide
   URI_part instead.

   Table 2 illustrates the definition of the information elements.  It
   provides examples using Apache log format strings [apache] when they



Bertrand, et al.        Expires February 11, 2013              [Page 15]


Internet-Draft                CDNI Logging                   August 2012


   exist.  The table is here for illustration and does not prescribe a
   specific encoding.

   +------------+------------------+-----------------------------------+
   | Name       | String           | Example                           |
   +------------+------------------+-----------------------------------+
   | Time       | %t               | [10/Oct/2000:13:55:36-0700]       |
   | Duration   | -                | -                                 |
   | Client-IP  | -                | -                                 |
   | Operation  | -                | -                                 |
   | URI_log    | -                | -                                 |
   | Protocol   | %H               | HTTP/1.0                          |
   | Request    | %m               | GET                               |
   | method     |                  |                                   |
   | Status     | %>s              | 200                               |
   | Bytes      | %O               | 2326                              |
   | transferre |                  |                                   |
   | d          |                  |                                   |
   | Bytes      | -                | -                                 |
   | received   |                  |                                   |
   | Header     | \"%{Referrer}i\" | "http://www.example.com/start.htm |
   |            | \"%{User-agent}i | l" "Mozilla/4.08 [en] (Win98; I   |
   |            | \"               |  ;Nav)"                           |
   +------------+------------------+-----------------------------------+

                   Table 2: Examples using Apache format

5.2.  Logging Record Information Elements for Content Delivery

   Table 3 details specific Logging fields that dCDN may provide to uCDN
   and that are related to content delivery operations.

   +-------------------+-----------------------------------------------+
   | Name              | Definition                                    |
   +-------------------+-----------------------------------------------+
   | uCDN-ID           | An element authenticating the operator of the |
   |                   | uCDN as the authority having delegated the    |
   |                   | request to the dCDN.                          |
   | Delivering-CDN-ID | An identifier (e.g., an aggregation of an IP  |
   |                   | address and a FQDN) of the Delivering CDN.    |
   |                   | The Delivering-CDN-ID might be considered as  |
   |                   | confidential by the dCDN.  In such case, the  |
   |                   | dCDN could either not provide this field to   |
   |                   | the uCDN or overwrite the Delivering-CDN-ID   |
   |                   | with its on identifier.                       |
   | End-User-IP       | The IP address of the client making a content |
   |                   | delivery request (or of its proxy).           |




Bertrand, et al.        Expires February 11, 2013              [Page 16]


Internet-Draft                CDNI Logging                   August 2012


   | Cache-bytes       | The number of body bytes served from caches.  |
   |                   | This quantity permits the computation of the  |
   |                   | byte hit ratio.                               |
   | Action            | The Action describes how a given request was  |
   |                   | treated locally: through which transport      |
   |                   | protocol, with or without content             |
   |                   | revalidation, with a cache hit or cache miss, |
   |                   | with fresh or stale content, and if relevant  |
   |                   | with which error.  Example with Squid format  |
   |                   | [squid]: "TCP_REFRESH_FAIL_HIT" means that an |
   |                   | expired copy of an object requested through   |
   |                   | TCP was in the cache.  Squid attempted to     |
   |                   | make an If-Modified-Since request, but it     |
   |                   | failed.  The old (stale) object was delivered |
   |                   | to the client.                                |
   +-------------------+-----------------------------------------------+

                  Table 3: Delivery Information Elements

5.3.  Logging Record Information Elements for

   Table 4 details specific Logging fields that are related to content
   acquisition operations.

   [Ed.  Note: split this section in two parts: logs provided by uCDN /
   logs provided by dCDN?]

   +------------+------------------------------------------------------+
   | Name       | Definition                                           |
   +------------+------------------------------------------------------+
   | dCDN       | An element authenticating the operator of the dCDN   |
   | identifier | as the authority requesting the content to the uCDN  |
   +------------+------------------------------------------------------+

                 Table 4: Acquisition Information Elements

   These information elements may be used in Content Acquisition Logging
   provided by dCDN to uCDN and potentially in Content Acquisition
   Logging provided by uCDN to dCDN.

5.4.  Logging Record Information Elements for Other Operations

   Logging can be used for debugging.  Therefore, all kind of CDN
   operations might be logged, depending on the agreement between the
   dCDN and the uCDN.  In particular, operations related to Request
   Routing, Metadata and Control interfaces can be logged.





Bertrand, et al.        Expires February 11, 2013              [Page 17]


Internet-Draft                CDNI Logging                   August 2012


6.  Core Logging Records

   This section defines a set of central events that a dCDN should
   register and publish through the Logging interface.

   We classify the logged events depending on the CDN operation to which
   they relate: Content Delivery, Content Acquisition, Content
   Invalidation/Purging, etc.

6.1.  Content Delivery

   Some CSPs pay a lot of attention to the protection of their content
   (e.g., premium video CSPs).  To fulfill the needs of these CSPs, a
   CDN shall log all the details of the content delivery authorizations.
   This means that a dCDN must be able to provide Logging detailing the
   content delivery/content acquisition authorizations and denials as
   well as information on why the request is authorized/denied.

   CSPs and CDSP pay a lot of attention to errors related to content
   delivery.  It is therefore of upmost importance that the dCDN
   provides detailed error information in the Logging data.  This
   information should typically be available even when Logging is
   aggregated (cf. Section 10.1).

   The content delivery events triggering the generation of a Logging
   Record include:

   o  Reception of a content request,

   The generated Logging Record typically embeds information about:

   o  Denial of delivery (error or unauthorized request) for a request,

   o  Beginning of delivery (authorization) of a requested content,

   o  End of an authorized delivery (success),

   o  End of an authorized delivery (failure).

6.2.  Content Acquisition

6.2.1.  Logging Records Provided by dCDN to uCDN

   When the uCDN requires the dCDN to provide Logging for acquisition
   related events, the events triggering the generation of a Logging
   Record include:





Bertrand, et al.        Expires February 11, 2013              [Page 18]


Internet-Draft                CDNI Logging                   August 2012


   o  Emission of a content acquisition request (first try or retry) for
      a cache hit or a cache miss with content revalidation

   The generated Logging Record typically embeds information about:

   o  Reception of a reply indicating denial of delivery (error or
      unauthorized request) for a content acquisition request,

   o  End of an authorized acquisition (success),

   o  End of an authorized acquisition (failure)

   Note that a dCDN may acquire content only from the uCDN.  It this
   case, the uCDN can log the dCDN's content acquisition operations
   itself, and thus, the uCDN may not require the dCDN to log
   acquisition related events (except for security or debugging
   reasons).

6.2.2.  Logging Records Provided by uCDN to dCDN

   When the dCDN requires the uCDN to provide Logging for acquisition
   related events, the events triggering the generation of a Logging
   Record include:

   o  Reception of a content acquisition request for the considered
      delivery service for a cache hit or a cache miss with content
      revalidation

   The generated Logging Record typically embeds information about:

   o  Emission of a reply indicating denial of delivery (error or
      unauthorized request) for a content acquisition request,

   o  End of an authorized acquisition (success),

   o  End of an authorized acquisition (failure).

6.3.  Content Invalidation and Purging

   When the uCDN requests a dCDN to log invalidation/purging events
   (e.g., for security), the events triggering the generation of a
   Logging Record include:

   o  Reception of a content invalidation/purging request

   The generated Logging Record typically embeds information about:





Bertrand, et al.        Expires February 11, 2013              [Page 19]


Internet-Draft                CDNI Logging                   August 2012


   o  Denial of the invalidation/purging request (error or unauthorized
      request),

   o  Beginning of invalidation/purging (authorization) for a given
      content purging request,

   o  End of an authorized invalidation/purging (success),

   o  End of an authorized invalidation/purging (failure).

6.4.  Logging Extensibility

   Future usages might introduce the need for additional Logging fields.
   In addition, some use-cases such as an Inter-Affiliate
   Interconnection [I-D.ietf-cdni-use-cases], might take advantage of
   extended Logging exchanges.  Therefore, it is important to permit
   CDNs to use additional Logging fields besides the standard ones, if
   they want.  For instance, an "Account-name" identifying the contract
   enforced by the dCDN for a given request could be provided in
   extended fields.

   The required Logging Records may depend on the considered services.
   For instance, static file delivery (e.g., pictures) typically does
   not include any delivery restrictions.  By contrast, video delivery
   typically implies strong content delivery restrictions, as explained
   in [I-D.ietf-cdni-use-cases], and Logging could include information
   about the enforcement of these restrictions.  Therefore, to ease the
   support of varied services as well as of future services, the Logging
   interface should support optional Logging Records.


7.  Default Logging Information Format

   Interconnected CDNs may support various Logging formats.  However,
   they must support at least the default Logging format described here.

7.1.  Logging Files

   [Ed.  Note: How many files (one per type of Delivery Service (e.g.,
   HTTP, WMP) and per type of Event (e.g., Errors, Delivery,
   Acquisition,...?)and what would be inside...  These aspects will be
   detailed in future versions.]

7.2.  File Format

   [Ed. note: The Logging file format is not necessarily independant of
   the selected transport protocol.  The definition of the Logging file
   format should be carried out consistently with the candidate protocol



Bertrand, et al.        Expires February 11, 2013              [Page 20]


Internet-Draft                CDNI Logging                   August 2012


   analysis for Logging transport.  The present content of this section
   is therefore non definitive.]

7.2.1.  Headers

   As initially proposed in [I-D.lefaucheur-cdni-logging-delivery],
   Logging files must include a header with the information described in
   Figure 3.


   +----------------+-------------------+------------------------------+
   | Field          | Description       | Examples                     |
   +----------------+-------------------+------------------------------+
   | Format         | Identification of | standard_cdni_errors_http_v1 |
   |                | CDNI Log format.  |                              |
   | Fields         | A description of  |                              |
   |                | the records format|                              |
   |                | (list of fields). |                              |
   | Log-ID         | Identifier        | abcdef1234                   |
   |                | for the CDNI Log  |                              |
   |                | file (facilitates |                              |
   |                | detection of      |                              |
   |                | duplicate Logs    |                              |
   |                | and tracking in   |                              |
   |                | case of           |                              |
   |                | aggregation).     |                              |
   | Log-Timestamp  | Time, in          | [20/Feb/2012:00:29.510+0200] |
   |                | milliseconds, the |                              |
   |                | CDNI Log was      |                              |
   |                | generated.        |                              |
   | Log-Origin     | Identifier of the | cdn1.cdni.example.com        |
   |                | authority (e.g.,  |                              |
   |                | dCDN or uCDN)     |                              |
   |                | providing the Log-|                              |
   |                | -ging             |                              |
   +----------------+-------------------+------------------------------+


                         Figure 3: Logging Headers

7.2.2.  Body (Logging Records) Format

   [Ed. note: the W3C extended log format is a good base candidate to
   look at.]

   [Ed.  Note: The format for Time is still to be agreed on.  RFC 5322
   (Section 3.3) format could be used or ISO 8601 formatted date and
   time in UTC (same format as proposed in



Bertrand, et al.        Expires February 11, 2013              [Page 21]


Internet-Draft                CDNI Logging                   August 2012


   [draft-caulfield-cdni-metadata-core-00]).  Also see RFC5424 Section 
   6.2.3.]

   [Ed.  Note: Records used for real time information and non-real time
   information could use different formats.]

7.2.3.  Footer Format

   As initially proposed in [I-D.lefaucheur-cdni-logging-delivery],
   Logging files must include a footer with the information described in
   Figure 4.


   +---------+----------------------------------------------+----------+
   | Field   | Description                                  | Examples |
   +---------+----------------------------------------------+----------+
   | Log     | Digest of the complete Log (facilitates      |          |
   | Digest  | detection of Log corruption)                 |          |
   +---------+----------------------------------------------+----------+


                         Figure 4: Logging footers


8.  Logging Format and Scope Negotiation

   [Ed.  Note: Format should be negotiated per delivery service]

   [Ed.  Note: uCDN shall be able to select the type of events that a
   dCDN should include in the Logging that the latter provides to the
   uCDN.]


9.  Logging Information Transport

   As presented in [I-D.ietf-cdni-problem-statement], several protocols
   already exist that could potentially be used to exchange CDNI Logging
   between interconnected CDNs.  The dCDN could publish non real-time
   Logging on a server where the uCDN would retrieve it using for
   example SSH File Transfer Protocol (SFTP).  If the CDNs need to
   exchange real-time information through the Logging interface, they
   could potentially rely on Web APIs, Syslog, SNMP...  The main
   criterion for selecting a Logging transport protocol is the time
   constraint for delivering the Logging.  Therefore, the present
   section highlights the candidate protocols for real-time and non
   real-time Logging exchanges.





Bertrand, et al.        Expires February 11, 2013              [Page 22]


Internet-Draft                CDNI Logging                   August 2012


9.1.  Major Requirements on Logging Protocols

   Logging data is sensitive as it provides the raw material for
   producing bills etc.  Therefore, the protocol delivering the Logging
   data must be reliable to avoid information loss.  In addition, the
   protocol must scale to support the transport of large amounts of
   Logging data.  Finally, this protocol must comply with the
   requirements identified in [I-D.ietf-cdni-requirements].

   CDNs need to trust Logging information, thus, they want to know:

   o  who issued the Logging (authentication), and

   o  if the Logging has been modified by a third party (integrity).

   This is extremely important, as the logs can provide a basis for
   accounting/billing.

   Logging also contains confidential data, and therefore, it should not
   be protected from eavesdropping.

   All these needs translate into security requirements on both the
   Logging data format and on the Logging protocol.

   [Ed. note: cf. requirements draft: "SEC-4 [MED] The CDNI solution
   should be able to ensure that the Downstream CDN cannot spoof a
   transaction log attempting to appear as if it corresponds to a
   request redirected by a given Upstream CDN when that request has not
   been redirected by this Upstream CDN.  This ensures non-repudiation
   by the Upstream CDN of transaction logs generated by the Downstream
   CDN for deliveries performed by the Downstream CDN on behalf of the
   Upstream CDN."]

9.2.  Recommended Logging Protocol for Non Real-Time Logging

   as explained in [I-D.ietf-cdni-problem-statement], "SNMP traps pose
   scalability concerns and SNMP does not support guaranteed delivery of
   Traps and therefore could result in log records being lost and the
   consequent CoDRs and billing records for that content delivery not
   being produced as well as that content delivery being invisible to
   any analytics platforms."

   [Ed.  Note: timing constraints... cf LOG-6 offline vs. constrained
   time / on demand access to real-time logging information]

   [Ed.  Note: in a later version, this memo will include an analysis of
   candidate protocols, based upon a set of (basic) requirements, such
   as reliable transport mode, preservation of the integrity of the



Bertrand, et al.        Expires February 11, 2013              [Page 23]


Internet-Draft                CDNI Logging                   August 2012


   information conveyed by the protocol, etc.]

   The offline exchange of non real-time Logging could rely on several
   protocols.  In particular, the dCDN could publish the Logging on a
   server where the uCDN would retrieve them using a secure protocol
   (yet to be identified).

   [Ed. note: event-triggered or periodic, why?]

   [Ed. note: Propose protocol and add call flow]

9.3.  Recommended Logging Protocol for Real-Time Logging

   The uCDN must be able to retrieve real-time information via near
   real-time methods such as: Syslog, SNMP, or through APIs, for
   example.

   [Ed. note: dCDN does not just forward requests for real time logging.
   It should probably provide other (more complex?) information in real
   time about the ongoing sessions (e.g., for every active session : IP
   of the client, service, CDN name, content consumed (full URL),
   average bit rate, downloaded size, date of session start?)


10.  Logging Process

   We walk through a "day in the life" of a CDN interconnection to
   present functions the two CDNs may require to exchange Logging
   information.  This will serve to illustrate many of the functions
   that could be supported through CDNI Logging interface.  We describe
   capabilities, such as log aggregation, anonymizing, and filtering,
   that might be added to CDNI in a later stage, to optimize Logging
   operations.

10.1.  Logging Aggregation

   CDNs typically handle millions of records per day.  The processing of
   these records to extract relevant monitoring and reporting
   information is expensive in terms of CPU and time.  Therefore, as
   stated in [I-D.ietf-cdni-framework], "a design tradeoff in the
   Logging interface is the degree of aggregation or summarization of
   data."

   In particular, dCDNs must aggregate the logs of their elements (e.g.,
   the Surrogates) to avoid both the complexity of distributing multiple
   log files to the uCDN and to avoid disclosing information about
   dCDN's internal topology.  This aggregation alleviates the Logging
   processing burden for the uCDN.



Bertrand, et al.        Expires February 11, 2013              [Page 24]


Internet-Draft                CDNI Logging                   August 2012


   Many situations also lead to the delivery of fragments of content
   (DASH, failure of delivery, partial delivery, PVR actions, etc.).  A
   dCDN may not publish a Logging Record for each piece of content it
   delivers, because this can lead to unacceptably large logs.  In
   particular, a Logging Record could provide aggregated information
   about the delivery of several content pieces. uCDN and dCDN must be
   able to agree on a level of granularity for the Logging Records.
   This problem is well described for the case of HTTP adaptive
   streaming in [I-D.ietf-cdni-framework] and
   [I-D.brandenburg-cdni-has].

   In the current version of the draft, we identify the following
   options that may be considered for reducing the amount of Logging
   data.

   o  Transmit only summaries, for instance, a summary may aggregate
      information of all deliveries that occur during a 5 minutes time
      slot or provide only Logging data related to content items that
      have been delivered at least a specific number of times.  Note
      that such aggregation leads to an information loss.  This may be
      problematic for some usages of Logging (e.g., debugging) and some
      information should always be present, for instance, information
      about content delivery errors (403,404,...).  The use multiple
      levels of Logging granularity such as in Apache (debug, notice,
      etc.) may help in providing the most relevant amount of
      information depending on the intended Logging usage, without
      having to renegotiate the Logging format.

   o  For HAS content, a way to compress logs with minimal information
      loss would be to merge all success 200 OK records Records related
      to the same level of video Quality into a single record with
      appropriate Start-time and End-time.  The only information lost in
      this process would be the Start-time and End-time for every video
      chunk.

   o  Losslessly compress the Logging data.

   o  Agree on a Logging retention duration and optionally on a maximum
      size of the Logging data that the dCDN must keep.  If this size is
      exceeded, the dCDN must alert the uCDN but may not keep more Logs
      for the considered time period.

   [Ed.  Note: cite Syslog's concepts for aggregation ]

10.2.  Logging Filtering

   The dCDN must be able to present only relevant information to the
   uCDN, to avoid unnecessary Logging processing load for the uCDN and



Bertrand, et al.        Expires February 11, 2013              [Page 25]


Internet-Draft                CDNI Logging                   August 2012


   potentially to protect End-Users' privacy.  Hence, the downstream CDN
   filters its logs, and passes the relevant records directly to each
   upstream CDN.  This requires that the downstream CDN can recognize
   the set of log entries that relate to each upstream CDN, for instance
   thanks to the "uCDN identifier" information element Table 3.

   The dCDN must be able to filter some internal scope data such as
   information related to its internal alarms (security, failures, load,
   etc).

   In some use cases described in [I-D.ietf-cdni-use-cases], the
   interconnected CDNs do not want to disclose details on their internal
   topology.  The dCDN must be able to filter confidential data on the
   dCDN's topology (number of servers, location, etc.).  In particular,
   information about the requests served by every Surrogate is
   confidential.  Therefore, the Logging information must be protected
   so that data such as Surrogates host-names is not disclosed to the
   uCDN.  In the "Inter-Affiliates Interconnection" use case, this
   information may be disclosed to the uCDN because both the dCDN and
   the uCDN are operated by entities of the same group.

10.3.  Logging Update and Rectification

   If Logging is generated periodically, it is important that the
   sessions that start in one Logging period and end in another are
   correctly reported.  If they are reported in the starting period,
   then the Logging of this period will be available only after the end
   of the session, which delays the Logging generation.

   A Logging rectification / update mechanism could be useful to reach a
   good trade-off between the Logging generation delay and the Logging
   accuracy.  Depending on the selected Logging protocol(s), such
   mechanism may be particularly invaluable for real time Logging, which
   must be provided rapidly and cannot wait for the end of operations in
   progress.


11.  Open Issues

   The level of granularity of the date/time information must be
   specified (clock accuracy).

   When to log the end of a session when the End-User pauses a video
   display?

   [Ed.  Note: check if all requirements are fulfilled by the proposed
   solution]




Bertrand, et al.        Expires February 11, 2013              [Page 26]


Internet-Draft                CDNI Logging                   August 2012


   [Ed. note: (comment from Kevin) how are errors handled ?  If the
   client gets handed a bunch of 403s and 404s, but still gets the
   content eventually, without triggering an event, are those still
   logged?  For Bytes-Transferred, if there were aborted requests, do
   those get counted as well?  Not all client behavior can be correlated
   with the simplified log.]


12.  IANA Considerations

   This memo includes no request to IANA.


13.  Security Considerations

13.1.  Privacy

   CDNs have the opportunity to collect detailed information about the
   downloads performed by End-Users.  The provision of this information
   to another CDN introduces End-Users privacy protection concerns.

13.2.  Non Repudiation

   Logging provides the raw material for charging.  It permits the dCDN
   to bill the uCDN for the content deliveries that the dCDN makes on
   behalf of the uCDN.  It also permits the uCDN to bill the CSP for the
   content delivery service.  Therefore, non-repudiation of Logging data
   is essential.  Some of the security issues and requirements on
   Logging are highlighted in Section 9.1.


14.  Acknowledgments

   The authors would like to thank Anne Marrec, Yannick Le Louedec, and
   Christian Jacquenet for detailed feedback on early versions of this
   document and for their input on existing Log formats.

   The authors would like also to thank Fabio Costa, Yvan Massot, Renaud
   Edel, and Joel Favier for their input and comments.

   Finally, they thank the contributors of the EU FP7 OCEAN project for
   valuable inputs.


15.  References






Bertrand, et al.        Expires February 11, 2013              [Page 27]


Internet-Draft                CDNI Logging                   August 2012


15.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

15.2.  Informative References

   [CLF]      A. Luotonen, "The Common Log-file Format, W3C (work in
              progress)", 1995, <http://www.w3.org/pub/WWW/Daemon/User/
              Config/Logging.html>.

   [ELF]      Phillip M. Hallam-Baker and Brian Behlendorf, "Extended
              Log File Format, W3C (work in progress), WD-logfile-
              960323", <http://www.w3.org/TR/WD-logfile.html>.

   [I-D.bertrand-cdni-experiments]
              Faucheur, F. and L. Peterson, "Content Distribution
              Network Interconnection (CDNI) Experiments",
              draft-bertrand-cdni-experiments-02 (work in progress),
              February 2012.

   [I-D.brandenburg-cdni-has]
              Brandenburg, R., Deventer, O., Faucheur, F., and K. Leung,
              "Models for adaptive-streaming-aware CDN Interconnection",
              draft-brandenburg-cdni-has-03 (work in progress),
              July 2012.

   [I-D.ietf-cdni-framework]
              Peterson, L. and B. Davie, "Framework for CDN
              Interconnection", draft-ietf-cdni-framework-01 (work in
              progress), July 2012.

   [I-D.ietf-cdni-problem-statement]
              Niven-Jenkins, B., Faucheur, F., and N. Bitar, "Content
              Distribution Network Interconnection (CDNI) Problem
              Statement", draft-ietf-cdni-problem-statement-08 (work in
              progress), June 2012.

   [I-D.ietf-cdni-requirements]
              Leung, K. and Y. Lee, "Content Distribution Network
              Interconnection (CDNI) Requirements",
              draft-ietf-cdni-requirements-03 (work in progress),
              June 2012.

   [I-D.ietf-cdni-use-cases]
              Bertrand, G., Emile, S., Burbridge, T., Eardley, P., Ma,
              K., and G. Watson, "Use Cases for Content Delivery Network
              Interconnection", draft-ietf-cdni-use-cases-10 (work in



Bertrand, et al.        Expires February 11, 2013              [Page 28]


Internet-Draft                CDNI Logging                   August 2012


              progress), August 2012.

   [I-D.lefaucheur-cdni-logging-delivery]
              Faucheur, F., Viveganandhan, M., and K. Leung, "CDNI
              Logging Formats for HTTP and HTTP Adaptive Streaming
              Deliveries", draft-lefaucheur-cdni-logging-delivery-01
              (work in progress), July 2012.

   [RFC3444]  Pras, A. and J. Schoenwaelder, "On the Difference between
              Information Models and Data Models", RFC 3444,
              January 2003.

   [RFC3466]  Day, M., Cain, B., Tomlinson, G., and P. Rzewski, "A Model
              for Content Internetworking (CDI)", RFC 3466,
              February 2003.

   [RFC3568]  Barbir, A., Cain, B., Nair, R., and O. Spatscheck, "Known
              Content Network (CN) Request-Routing Mechanisms",
              RFC 3568, July 2003.

   [apache]   "Apache 2.2 log files documentation", Feb. 2012,
              <http://httpd.apache.org/docs/current/logs.html>.

   [squid]    "Squid Log-Format documentation", Feb. 2012,
              <http://wiki.squid-cache.org/SquidFaq/SquidLogs>.


Appendix A.  Examples Log Format

   This section provides example of log formats implemented in existing
   CDNs, web servers, and caching proxies.

   Web servers (e.g., Apache) maintain at least one log file for logging
   accesses to content (the Access Log).  They can typically be
   configured to log errors in a separate log file (the Error Log).  The
   log formats can be specified in the server's configuration files.
   However, webmasters often use standard log formats to ease the log
   processing with available log analysis tools.

A.1.  W3C Common Log File (CLF) Format

   The Common Log File (CLF) format defined by the World Wide Web
   Consortium (W3C) working group is compatible with many log analysis
   tools and is supported by the main web servers (e.g., Apache) Access
   Logs.

   According to [CLF], the common log-file format is as follows:
   remotehost rfc931 authuser [date] "request" status bytes.



Bertrand, et al.        Expires February 11, 2013              [Page 29]


Internet-Draft                CDNI Logging                   August 2012


   Example (from [apache]): 127.0.0.1 - frank [10/Oct/2000:13:55:36
   -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326

   The fields are defined as follows [CLF]:

   +------------+------------------------------------------------------+
   | Element    | Definition                                           |
   +------------+------------------------------------------------------+
   | remotehost | Remote hostname (or IP number if DNS hostname is not |
   |            | available, or if DNSLookup is Off.                   |
   | rfc931     | The remote logname of the user.                      |
   | authuser   | The username that the user employed to authenticate  |
   |            | himself.                                             |
   | [date]     | Date and time of the request.                        |
   | "request"  | An exact copy of the request line that came from the |
   |            | client.                                              |
   | status     | The status code of the HTTP reply returned to the    |
   |            | client.                                              |
   | bytes      | The content-length of the document transferred.      |
   +------------+------------------------------------------------------+

                Table 5: Information elements in CLF format

A.2.  W3C Extended Log File (ELF) Format

   The Extended Log File (ELF) format defined by W3C extends the CLF
   with new fields.  This format is supported by Microsoft IIS 4.0 and
   5.0.

   The supported fields are listed below [ELF].





















Bertrand, et al.        Expires February 11, 2013              [Page 30]


Internet-Draft                CDNI Logging                   August 2012


    +------------+---------------------------------------------------+
    | Element    | Definition                                        |
    +------------+---------------------------------------------------+
    | date       | Date at which transaction completed               |
    | time       | Time at which transaction completed               |
    | time-taken | Time taken for transaction to complete in seconds |
    | bytes      | bytes transferred                                 |
    | cached     | Records whether a cache hit occurred              |
    | ip         | IP address and port                               |
    | dns        | DNS name                                          |
    | status     | Status code                                       |
    | comment    | Comment returned with status code                 |
    | method     | Method                                            |
    | uri        | URI                                               |
    | uri-stem   | Stem portion alone of URI (omitting query)        |
    | uri-query  | Query portion alone of URI                        |
    +------------+---------------------------------------------------+

                Table 6: Information elements in ELF format

   Some fields start with a prefix (e.g., "c-", "s-"), which explains
   which host (client/server/proxy) the field refers to.

   o  Prefix Description

   o  c- Client

   o  s- Server

   o  r- Remote

   o  cs- Client to Server.

   o  sc- Server to Client.

   o  sr- Server to Remote Server (used by proxies)

   o  rs- Remote Server to Server (used by proxies)

   Example: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-
   username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status
   time-taken

   2011-11-23 15:22:01 x.x.x.x GET /file 80 y.y.y.y Mozilla/
   5.0+(Windows;+U;+Windows+NT+6.1;+en-US;+rv:1.9.1.6)+Gecko/
   20091201+Firefox/3.5.6+GTB6 200 0 0 2137





Bertrand, et al.        Expires February 11, 2013              [Page 31]


Internet-Draft                CDNI Logging                   August 2012


A.3.  National Center for Supercomputing Applications (NCSA) Common Log
      Format

   This format for Access Logs offers the following fields:

   o  host rfc931 date:time "request" statuscode bytes

   o  x.x.x.x userfoo [10/Jan/2010:21:15:05 +0500] "GET /index.html
      HTTP/1.0" 200 1043

A.4.  NCSA Combined Log Format

   The NCSA Combined log format is an extension of the NCSA Common log
   format with three (optional) additional fields: the referral field,
   the user_agent field, and the cookie field.

   o  host rfc931 username date:time request statuscode bytes referrer
      user_agent cookie

   o  Example: x.x.x.x - userfoo [21/Jan/2012:12:13:56 +0500] "GET
      /index.html HTTP/1.0" 200 1043 "http://www.example.com/" "Mozilla/
      4.05 [en] (WinNT; I)" "USERID=CustomerA;IMPID=01234"

A.5.  NCSA Separate Log Format

   The NCSA Separate log format refers to a log format in which the
   information gathered is separated into three separate files.  This
   way, every entry in the Access Log (in the NCSA Common log format) is
   complemented with an entry in a Referral log and another one in an
   Agent log.  These three entries can be correlated easily thanks to
   the date:time value.  The format of the Referral log is as follows:

   o  date:time referrer

   o  Example: [21/Jan/2012:12:13:56 +0500]
      "http://www.example.com/index.html"

   The format of the Agent log is as follows:

   o  date:time agent

   o  [21/Jan/2012:12:13:56 +0500] "Microsoft Internet Explorer - 5.0"

A.6.  Squid 2.0 Native Log Format for Access Logs

   Squid [squid] is a popular piece of open-source software for
   transforming a Linux host into a caching proxy.  Variations of Squid
   log format are supported by some CDNs.



Bertrand, et al.        Expires February 11, 2013              [Page 32]


Internet-Draft                CDNI Logging                   August 2012


   Squid common access log format is as follow: time elapsed remotehost
   code/status bytes method URL rfc931 peerstatus/peerhost type.

   Squid also supports a more detailed native access log format:
   Timestamp Elapsed Client Action/Code Size Method URI Ident Hierarchy/
   From Content

   According to Squid 2.0 documentation [squid], these fields are
   defined as follows:

   +-----------+-------------------------------------------------------+
   | Element   | Definition                                            |
   +-----------+-------------------------------------------------------+
   | time      | Unix timestamp as UTC seconds with a millisecond      |
   |           | resolution.                                           |
   | duration  | The elapsed time in milliseconds the transaction      |
   |           | busied the cache.                                     |
   | client    | The client IP address.                                |
   | address   |                                                       |
   | bytes     | The size is the amount of data delivered to the       |
   |           | client, including headers.                            |
   | request   | The request method to obtain an object.               |
   | method    |                                                       |
   | URL       | The requested URL.                                    |
   | rfc931    | may contain the ident lookups for the requesting      |
   |           | client (turned off by default)                        |
   | hierarchy | The hierarchy information provides information on how |
   | code      | the request was handled (forwarding it to another     |
   |           | cache, or requesting the content to the Origin        |
   |           | Server).                                              |
   | type      | The content type of the object as seen in the HTTP    |
   |           | reply header.                                         |
   +-----------+-------------------------------------------------------+

               Table 7: Information elements in Squid format

   Squid also uses a "store log", which covers the objects currently
   kept on disk or removed ones, for debugging purposes typically.













Bertrand, et al.        Expires February 11, 2013              [Page 33]


Internet-Draft                CDNI Logging                   August 2012


Authors' Addresses

   Gilles Bertrand (editor)
   France Telecom - Orange
   38-40 rue du General Leclerc
   Issy les Moulineaux,   92130
   FR

   Phone: +33 1 45 29 89 46
   Email: gilles.bertrand@orange.com


   Stephan Emile
   France Telecom - Orange
   2 avenue Pierre Marzin
   Lannion  F-22307
   France

   Email: emile.stephan@orange.com


   Roy Peterkofsky
   Skytide, Inc.
   One Kaiser Plaza, Suite 785
   Oakland  CA 94612
   USA

   Phone: +01 510 250 4284
   Email: roy@skytide.com






















Bertrand, et al.        Expires February 11, 2013              [Page 34]


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/