--- 1/draft-ietf-cdni-logging-11.txt 2014-07-04 07:14:38.917593246 -0700 +++ 2/draft-ietf-cdni-logging-12.txt 2014-07-04 07:14:39.009595548 -0700 @@ -1,22 +1,22 @@ Internet Engineering Task Force F. Le Faucheur, Ed. Internet-Draft Cisco Systems Intended status: Standards Track G. Bertrand, Ed. -Expires: September 26, 2014 I. Oprescu, Ed. +Expires: January 5, 2015 I. Oprescu, Ed. Orange R. Peterkofsky Skytide, Inc. - March 25, 2014 + July 4, 2014 CDNI Logging Interface - draft-ietf-cdni-logging-11 + draft-ietf-cdni-logging-12 Abstract This memo specifies the Logging interface between a downstream CDN (dCDN) and an upstream CDN (uCDN) that are interconnected as per the CDN Interconnection (CDNI) framework. First, it describes a reference model for CDNI logging. Then, it specifies the CDNI Logging File format and the actual protocol for exchange of CDNI Logging Files. @@ -28,21 +28,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on September 26, 2014. + This Internet-Draft will expire on January 5, 2015. Copyright Notice Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -69,44 +69,48 @@ 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 2.2.5.6. Notions common to multiple Log Consuming Applications . . . . . . . . . . . . . . . . . . 13 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 3.3. CDNI Logging File Directives . . . . . . . . . . . . . . 19 - 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 22 + 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 23 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 23 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 32 - 4. CDNI Logging File Exchange Protocol . . . . . . . . . . . . . 32 - 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 33 - 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 33 - 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 34 - 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 34 - 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 35 - 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 37 - 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38 - 5.1. CDNI Logging Directive Names Registry . . . . . . . . . . 38 - 5.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 39 - 5.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 40 - 5.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 41 - 6. Security Considerations . . . . . . . . . . . . . . . . . . . 41 - 6.1. Authentication, Confidentiality, Integrity Protection . . 41 - 6.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 42 - 6.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 42 - 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 43 - 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 44 - 8.1. Normative References . . . . . . . . . . . . . . . . . . 44 - 8.2. Informative References . . . . . . . . . . . . . . . . . 44 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 46 + 3.6. Cascaded CDNI Logging Files Example . . . . . . . . . . . 34 + 4. Protocol for Exchange of CDNI Logging File After Full + Collection . . . . . . . . . . . . . . . . . . . . . . . . . 37 + 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 38 + 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 38 + 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 38 + 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 39 + 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 39 + 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 41 + 5. Protocol for Exchange of CDNI Logging File During Collection 42 + 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43 + 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 43 + 6.2. CDNI Logging Record-Types Registry . . . . . . . . . . . 43 + 6.3. CDNI Logging Field Names Registry . . . . . . . . . . . . 44 + 6.4. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 46 + + 7. Security Considerations . . . . . . . . . . . . . . . . . . . 46 + 7.1. Authentication, Confidentiality, Integrity Protection . . 46 + 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 47 + 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 47 + 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 48 + 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 48 + 9.1. Normative References . . . . . . . . . . . . . . . . . . 48 + 9.2. Informative References . . . . . . . . . . . . . . . . . 49 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 50 1. Introduction This memo specifies the CDNI Logging interface between a downstream CDN (dCDN) and an upstream CDN (uCDN). First, it describes a reference model for CDNI logging. Then, it specifies the CDNI Logging File format and the actual protocol for exchange of CDNI Logging Files. The reader should be familiar with the following documents: @@ -308,21 +312,22 @@ ===> CDNI Logging Interface ***> outside the scope of CDNI Figure 1: Interactions in CDNI Logging Reference Model A dCDN (e.g., dCDN-2) integrates the relevant Logging information obtained from its dCDNs (i.e., dCDN-3) in the Logging information that it provides to the uCDN, so that the uCDN ultimately obtains all Logging information relevant to a CSP for which it acts as the - authoritative CDN. + authoritative CDN. Such aggregation is further discussed in + Section 3.6. Note that the format of Logging information that a CDN provides over the CDNI interface might be different from the one that the CDN uses internally. In this case, the CDN needs to reformat the Logging information before it provides this information to the other CDN over the CDNI Logging interface. Similarly, a CDN might reformat the Logging information that it receives over the CDNI Logging interface before injecting it into its log-consuming applications or before providing some of this Logging information to the CSP. Such reformatting operations introduce latency in the logging distribution @@ -387,26 +392,26 @@ 2.2.1. Logging Generation and During-Generation Aggregation CDNs typically generate Logging information for all significant task completions, events, and failures. Logging information is typically generated by many devices in the CDN including the surrogates, the request routing system, and the control system. The amount of Logging information generated can be huge. Therefore, during contract negotiations, interconnected CDNs often agree on a retention duration for Logging information, and/or potentially on a - maximum volume of Logging information that the dCDN must keep. If - this volume is exceeded, the dCDN is expected to alert the uCDN but - may not keep more Logging information for the considered time period. - In addition, CDNs may aggregate Logging information and transmit only - summaries for some categories of operations instead of the full - Logging information. Note that such aggregation leads to an + maximum volume of Logging information that the dCDN ought to keep. + If this volume is exceeded, the dCDN is expected to alert the uCDN + but may not keep more Logging information for the considered time + period. In addition, CDNs may aggregate Logging information and + transmit only summaries for some categories of operations instead of + the full Logging information. Note that such aggregation leads to an information loss, which may be problematic for some usages of the Logging information (e.g., debugging). [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In accordance with the recommendations articulated there, it is expected that a surrogate will generate separate Logging information for delivery of each chunk of HAS content. This ensures that separate Logging information can then be provided to interconnected CDNs over the CDNI Logging interface. Still in line with the recommendations of [RFC6983], the Logging information for per-chunck delivery may @@ -455,54 +460,55 @@ The CDN will also filter some internal scope information such as information related to its internal alarms (security, failures, load, etc). In some use cases described in [RFC6770], the interconnected CDNs do not want to disclose details on their internal topology. The filtering process can then also filter confidential data on the dCDNs' topology (number of servers, location, etc.). In particular, information about the requests served by each Surrogate may be - confidential. Therefore, the Logging information must be protected - so that data such as Surrogates' hostnames are not disclosed to the - uCDN. In the "Inter-Affiliates Interconnection" use case, this - information may be disclosed to the uCDN because both the dCDN and - the uCDN are operated by entities of the same group. + confidential. Therefore, the Logging information needs to be + protected so that data such as Surrogates' hostnames are not + disclosed to the uCDN. In the "Inter-Affiliates Interconnection" use + case, this information may be disclosed to the uCDN because both the + dCDN and the uCDN are operated by entities of the same group. 2.2.4. Logging Rectification and Post-Generation Aggregation If Logging information is generated periodically, it is important that the sessions that start in one Logging period and end in another are correctly reported. If they are reported in the starting period, then the Logging information of this period will be available only after the end of the session, which delays the Logging information - generation. + generation. A simple approach is to provide the complete Logging + Record for a session in the Logging Period of the session end. A Logging rectification/update mechanism could be useful to reach a good trade-off between the Logging information generation delay and the Logging information accuracy. In the presence of HAS, some log-consuming applications can benefit from aggregate per-session logs. For example, for analytics, per- session logs allow display of session-related trends which are much more meaningful for some types of analysis than chunk-related trends. In the case where aggregate logs have been generated directly by the log-generating entities, those can be used by the applications. In the case where aggregate logs have not been generated, the Rectification process can be extended with a Post-Generation Aggregation process that generates per-session logs from the per- chunk logs, possibly leveraging the information included in the per- chunk logs for that purpose (Content Collection IDentifier and a Session IDentifier). However, in accordance with [RFC6983], this document does not define exchange of such aggregate logs on the CDNI Logging interface. We note that this is for further study and - outside the scope of this document.. + outside the scope of this document. 2.2.5. Log-Consuming Applications 2.2.5.1. Maintenance/Debugging Logging information is useful to permit the detection (and limit the risk) of content delivery failures. In particular, Logging information facilitates the detection of configuration issues. To detect faults, Logging information needs to report success and @@ -595,22 +601,22 @@ o Number of delivery requests received from End Users in a given region for each piece of content, during a given period of time (e.g., hour/day/week/month) o Percentage of delivery successes/failures among the aforementioned requests o Number of failures listed by failure type (e.g., HTTP error code) for requests received from End Users in a given region and for - each piece of content, during a given period of time (e.g., hour/ - day/week/month) + each piece of content, during a given period of time (e.g., + hour/day/week/month) o Number and cause of premature delivery termination for End Users in a given region and for each piece of content, during a given period of time (e.g., hour/day/week/month) o Maximum and mean number of simultaneous sessions established by End Users in a given region, for a given Content Provider, and during a given period of time (e.g., hour/day/week/month) o Volume of traffic delivered for sessions established by End Users @@ -768,22 +774,22 @@ | | | | | #Directive P+Q | +----------------------------------------------------------+ Figure 3: Structure of Logging Files The CDNI Logging File format is inspired from the W3C Extended Log File Format [ELF]. However, it is fully specified by the present document. Where the present document differs from the W3C Extended - Log File Format, an implementation of CDNI Logging MUST comply with - the present document. + Log File Format, an implementation of the CDNI Logging interface MUST + comply with the present document. Using a format that resembles the W3C Extended Log File Format is intended to keep CDNI logging format close to the intra-CDN Logging information format commonly used in CDNs today, thereby minimizing systematic translation at CDN/CDNI boundary. A CDNI Logging File MUST contain a sequence of lines containing US- ASCII characters [CHAR_SET] terminated by CRLF. Each line of a CDNI Logging File MUST contain either a directive or a @@ -808,25 +814,25 @@ = 1* 3.3. CDNI Logging File Directives The CDNI Logging File directives are defined by the following rules: directive = DIRNAME ":" HTAB DIRVAL DIRNAME = + Logging Directive Names registry (Section 6.1)> DIRVAL = + (Section 6.1) for the corresponding DIRNAME> An implementation of the CDNI Logging interface MUST support all of the following directives, listed below by their directive name: o Version: * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT * directive value: indicates the version of the CDNI Logging File format. The value MUST be "CDNI/1.0" for the version specified @@ -873,103 +879,107 @@ responsible for transmitting the CDNI Logging File (e.g., the dCDN). * occurrence: there MUST be zero or exactly one instance of this directive per CDNI Logging File. This directive MAY be added by the uCDN (e.g., before storing the CDNI Logging File). It MUST NOT be included by the dCDN. The mechanisms used by the uCDN to establish and validate the entity responsible for the CDNI Logging File is outside the scope of the present document. We observe that, in particular, this may be achieved through - authentication mechanisms that are part of the CDNI Logging - File pull mechanism (Section 4.2). + authentication mechanisms that are part of the transport layer + of the CDNI Logging File pull mechanism (Section 4.2). o Record-Type: * format: NAMEFORMAT * directive value: indicates the type of the CDNI Logging Records that follow this directive, until another Record-Type directive (or the end of the CDNI Logging File). This can be any CDNI Logging Record type registered in the CDNI Logging Record-types - registry (Section 5.2). "cdni_http_request_v1" MUST be - indicated as the Record-Type directive value for CDNI Logging - records corresponding to HTTP requests (e.g., a HTTP delivery - request) as specified in Section 3.4.1. + registry (Section 6.2). For example this may be + "cdni_http_request_v1" as specified in Section 3.4.1. * occurrence: there MUST be at least one instance of this directive per CDNI Logging File. The first instance of this directive MUST precede a Fields directive and MUST precede all CDNI Logging Records. o Fields: * format: FIENAME * ; where FIENAME can take any CDNI Logging field name registered in the CDNI Logging Field - Names registry (Section 5.3). + Names registry (Section 6.3). * directive value: this lists the names of all the fields for which a value is to appear in the CDNI Logging Records that follow the instance of this directive (until another instance of this directive). The names of the fields, as well as their occurrences, MUST comply with the corresponding rules specified in the document referenced in the CDNI Logging Record-types - registry (Section 5.2) for the corresponding CDNI Logging + registry (Section 6.2) for the corresponding CDNI Logging Record-Type. * occurrence: there MUST be at least one instance of this directive per Record-Type directive. The first instance of this directive for a given Record-Type MUST appear before any - CDNI Logging Record for this Record-Type. + CDNI Logging Record for this Record-Type. One situation where + more than one instance of the Fields directive can appear + within a given CDNI Logging File, is when there is a change, in + the middle of a fairly large logging period, in the agreement + between the uCDN and the dCDN about the set of Fields that are + to be exchanged. The multiple occurences allow records with + the old set of fields and records with the new set of fields to + be carried inside the same Logging File. o Integrity-Hash: * format: 32HEXDIG * directive value: This directive permits the detection of a corrupted CDNI Logging File. This can be useful, for instance, if a problem occurs on the filesystem of the dCDN Logging system and leads to a truncation of a logging file. The valid Integrity-Hash value is included in this directive by the entity that transmits the CDNI Logging File. It is computed by applying the MD5 ([RFC1321]) cryptographic hash function on the CDNI Logging File, including all the directives and logging - records, up to the Integrrity-Hash directive itself, excluding + records, up to the Integrity-Hash directive itself, excluding the Integrity-Hash directive itself. The Integrity-Hash value is represented as a US-ASCII encoded hexadecimal number, 32 digits long (representing a 128 bit hash value). The entity receiving the CDNI Logging File also computes in a similar way the MD5 hash on the received CDNI Logging File and compares this hash to the value of the Integrity-Hash directive. If the two values are equal, then the received CDNI Logging File MUST - be considered non-corrupted. Note that this is not a guarantee - that the file has not been tampered with by a third party. If - the two values are different, the received CDNI Logging File - MUST be considered corrupted. The behavior of the entity that - received a corrupted CDNI Logging File is outside the scope of - this specification; we note that the entity MAY attempt to pull - again the same CDNI Logging File from the transmitting entity. - If the entity receiving a non-corrupted CDNI Logging File adds - a Verified-Origin directive, it MUST then recompute and update - the Integrity-Hash directive so it also protects the added + be considered non-corrupted. If the two values are different, + the received CDNI Logging File MUST be considered corrupted. + The behavior of the entity that received a corrupted CDNI + Logging File is outside the scope of this specification; we + note that the entity MAY attempt to pull again the same CDNI + Logging File from the transmitting entity. If the entity + receiving a non-corrupted CDNI Logging File adds a Verified- + Origin directive, it MUST then recompute and update the + Integrity-Hash directive so it also protects the added Verified-Origin directive. * occurrence: there MUST be zero or exactly one instance of this directive. There SHOULD be exactly one instance of this directive. One situation where that directive could be omitted is where integrity protection is already provided via another mechanism (for example if an integrity hash is associated to - the CDNI Logging File out of band through the CDNI Logging - Logging Feed Section 4.1 leveraging ATOM extensions such as - those proposed in [I-D.snell-atompub-link-extensions]. When - present, this field MUST be the last line of the CDNI Logging - File. + the CDNI Logging File out of band through the CDNI Logging Feed + ( Section 4.1) leveraging ATOM extensions such as those + proposed in [I-D.snell-atompub-link-extensions]. When present, + the Integrity-Hash field MUST be the last line of the CDNI + Logging File. 3.4. CDNI Logging Records A CDNI Logging Record consists of a sequence of CDNI Logging Fields relating to that single CDNI Logging Record. CDNI Logging Fields MUST be separated by the "horizontal tabulation (HTAB)" character. To facilitate readability, a prefix scheme is used for CDNI Logging @@ -987,31 +997,31 @@ o u: refers to the uCDN (relative to a given CDN acting as a dCDN) o cs: refers to communication from the User Agent towards the dCDN Surrogate o sc: refers to communication from the dCDN Surrogate towards the User Agent An implementation of the CDNI Logging interface as per the present - specification MUST support the CDNI HTTP Delivery Records as + specification MUST support the CDNI HTTP Request Logging Record as specified in Section 3.4.1. A CDNI Logging Record is defined by the following rules: FIEVAL = = FIEVAL * ; where FIEVAL - contains the CDNI Logging field values corresponding to the CDNI + contains the CDNI Logging field value corresponding to the CDNI Logging field names (FIENAME) listed is the last Fields directive - predecing the present CDNI Logging Record. + preceding the present CDNI Logging Record. 3.4.1. HTTP Request Logging Record This section defines the CDNI Logging Record of Record-Type "cdni_http_request_v1". It is applicable to content delivery performed by the dCDN using HTTP/1.0([RFC1945]), HTTP/1.1([RFC2616]) or HTTPS ([RFC2818]). We observe that, in the case of HTTPS delivery, there may be value in logging additional information specific to the operation of HTTP over TLS and we note that this is outside the scope of the present document and may be addressed in a @@ -1126,22 +1137,23 @@ * field value: the destination TCP port (i.e., the "server" port) in the request received by the Surrogate. * occurrence: there MUST be zero or exactly one instance of this field. o cs-method: * format: NHTABSTRING - * field value: this is the HTTP method of the HTTP request - received by the Surrogate. + * field value: this is the method of the request received by the + Surrogate. In the case of HTTP delivery, this is the HTTP + method in the request. * occurrence: There MUST be one and only one instance of this field. o cs-uri: * format: NHTABSTRING * field value: this is the complete URL of the request received by the Surrogate. It is exactly in the format of a http_URL @@ -1181,41 +1193,44 @@ specified in [RFC2616] of the Request-Line of the request received by the Surrogate (e.g., "HTTP/1.1"). * occurrence: there MUST be one and only one instance of this field. o sc-status: * format: 3DIGIT - * field value: this is the HTTP Status-Code in the HTTP response - from the Surrogate. + * field value: this is the Status-Code in the response from the + Surrogate. tIn the case of HTTP delivery, this is the HTTP + Status-Code in the HTTP response. * occurrence: There MUST be one and only one instance of this field. o sc-total-bytes: * format: 1*DIGIT - * field value: this is the total number of bytes of the HTTP - response sent by the Surrogate in response to the request. - This includes the bytes of the Status-Line, the bytes of the - HTTP headers and the bytes of the message-body. + * field value: this is the total number of bytes of the response + sent by the Surrogate in response to the request. In the case + of HTTP delivery, this includes the bytes of the Status-Line, + the bytes of the HTTP headers and the bytes of the message- + body. * occurrence: There MUST be one and only one instance of this field. o sc-entity-bytes: * format: 1*DIGIT + * field value: this is the number of bytes of the message-body in the HTTP response sent by the Surrogate in response to the request. This does not include the bytes of the Status-Line or the bytes of the HTTP headers. * occurrence: there MUST be zero or exactly one instance of this field. o cs(): @@ -1395,88 +1411,261 @@ them. An uCDN-side implementation of the CDNI Logging interface MUST be able to accept CDNI Logging Files with CDNI Logging Records of Record-Type "cdni_http_request_v1" containing any CDNI Logging Field defined in Section 3.4.1 as long as the CDNI Logging Record and the CDNI Logging File are compliant with the present document. 3.5. CDNI Logging File Example + Let us consider the upstream CDN and the downstream CDN labelled uCDN + and dCDN-1 in Figure 1. When dCDN-1 acts as a downstream CDN for + uCDN and performs content delivery on behalf of uCDN, dCDN-1 will + include the CDNI Logging Records corresponding to the content + deliveries performed on behalf of uCDN in the CDNI Logging Files for + uCDN. An example CDNI Logging File communicated by dCDN-1 to uCDN is + shown below in Figure 4. + #Version:CDNI/1.0 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 - #Claimed-Origin:cdni-logging-entity.dcdn.example.com + #Claimed-Origin:cdni-logging-entity.dcdn-1.example.com #Record-Type:cdni_http_request_v1 - #Fields:datetimetime-takenc-ipcs- - methodu-uriprotocolsc-statussc-total- - bytescs(User-Agent)cs(Referer)s-cached + #Fields:datetimetime-takenc-ip + cs-methodu-uriprotocolsc-status + sc-total-bytescs(User-Agent)cs(Referer) + s-cached - 2013-05-1700:38:06.8259.05810.5.7.1GETh - ttp://cdni-ucdn.dcdn.example.com/video/movie100.mp4HTTP/ - 1.12006729891"Mozilla/5.0 (Windows; U; Windows NT - 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 - Safari /533.4""host1.example.com"1 + 2013-05-1700:38:06.8259.05810.5.7.1GET + http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 + HTTP/1.12006729891"Mozilla/5.0 + (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like + Gecko) Chrome/5.0.375.127 Safari/533.4" + "host1.example.com"1 2013-05-1700:39:09.14515.3210.5.10.5GET - http://cdni-ucdn.dcdn.example.com/video/movie118.mp4HTTP/ - 1.120015799210"Mozilla/5.0 (Windows; U; Windows NT - 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 - Safari /533.4""host1.example.com"1 + http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 + HTTP/1.120015799210"Mozilla/5.0 + (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like + Gecko) Chrome/5.0.375.127 Safari/533.4" + "host1.example.com"1 - 2013-05-1700:42:53.43752.87910.5.10.5GEThttp://cdni-ucdn.dcdn.example.com/video/picture11.mp4HTTP/ - 1.020097234724"Mozilla/5.0 (Windows; U; Windows NT - 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 - Safari /533.4""host5.example.com"0 + 2013-05-1700:42:53.43752.87910.5.10.5GET + http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 + HTTP/1.020097234724"Mozilla/5.0 + (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like + Gecko) Chrome/5.0.375.127 Safari/533.4" + "host5.example.com"0 #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e -4. CDNI Logging File Exchange Protocol + [Editor's Note: compute the correct Integrity-Hash value once example + is frozen] - This document specifies a protocol for the exchange of CDNI Logging - Files as specified in Section 3. + Figure 4: CDNI Logging File Example + + If uCDN establishes by some means (e.g. via TLS authentication when + pulling the CDNI Logging File) the identity of the entity from which + it pulled the CDNI Logging File, uCDN can add to the CDNI Logging a + Verified-Origin directive as illustrated below: + + #Verified-Origin:cdni-logging-entity.dcdn-1.example.com + As illustrated in Figure 2, uCDN will then ingest the corresponding + CDNI Logging Records into its Collection process, alongside the + Logging Records generated locally by the uCDN itself. This allows + uCDN to aggregate Logging Records for deliveries performed by itself + (through Records generated locally) as well as for deliveries + performed by its downstream CDN(s). This aggregate information can + then be used (after Filtering and Rectification, as illustrated in + Figure 2) by Log Consuming Applications that take into account + deliveries performed by uCDN as well as by all of its downstream + CDNs. + + We observe that the time between + + 1. when a delivery is completed in dCDN and + + 2. when the corresponding Logging Record is ingested by the + Collection process in uCDN + + depends on a number of parameters such as the Logging Period agreed + to by uCDN and dCDN, how much time uCDN waits before pulling the CDNI + Logging File once it is advertised in the CDNI Logging Feed, and the + time to complete the pull of the CDNI Logging File. Therefore, if we + consider the set of Logging Records aggregated by the Collection + process in uCDN in a given time interval, there could be a permanent + significant timing difference between the CDNI Logging Records + received from the dCDN and the Logging Records generated locally. + For example, in a given time interval, the Collection process in uCDN + may be aggregating Logging Records generated locally by uCDN for + deliveries performed in the last hour and CDNI Logging Records + generated in the dCDN for deliveries in the hour before last. + +3.6. Cascaded CDNI Logging Files Example + + Let us consider the cascaded CDN scenario of uCDN, dCDN-2 and dCDN-3 + as depicted in Figure 1. After completion of a delivery by dCDN-3 on + behalf of dCDN-2, dCDN-3 will include a corresponding Logging Record + in a CDNI Logging File that will be pulled by dCDN-2 and that is + illustrated below in Figure 5. In practice, a CDNI Logging File is + likely to contain a very high number of CDNI Logging Records. + However, for readability, the example in Figure 5 contains a single + CDNI Logging Record. + + #Version:CDNI/1.0 + + #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde + + #Claimed-Origin:cdni-logging-entity.dcdn-3.example.com + + #Record-Type:cdni_http_request_v1 + + #Fields:datetimetime-takenc-ip + cs-methodu-uriprotocolsc-status + sc-total-bytescs(User-Agent)cs(Referer) + s-cached + + 2013-05-1700:39:09.11914.0710.5.10.9GET + http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 + HTTP/1.120015799210"Mozilla/5.0 + (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like + Gecko) Chrome/5.0.375.127 Safari /533.4" + "host1.example.com"1 + + #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e + + [Editor's Note: compute the correct Integrity-Hash value once example + is frozen] + + Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) + + If dCDN-2 establishes by some means (e.g. via TLS authentication when + pulling the CDNI Logging File) the identity of the entity from which + it pulled the CDNI Logging File, dCDN-2 can add to the CDNI Logging a + Verified-Origin directive as illustrated below: + + #Verified-Origin:cdni-logging-entity.dcdn-3.example.com + + dCDN-2 (behaving as an upstream CDN from the viewpoint of dCDN-3) + will then ingest the CDNI Logging Record for the considered dCDN-3 + delivery into its Collection process (as illustrated in Figure 2). + This Logging Record may be aggregated with Logging Records generated + locally by dCDN-2 for deliveries performed by dCDN-2 itself. Say, + for illustration, that the content delivery performed by dCDN-3 on + behalf of dCDN-2 had actually been redirected to dCDN-2 by uCDN, and + say that another content delivery has just been redirected by uCDN to + dCDN-2 and that dCDN-2 elected to perform the corresponding delivery + itself. Then after Filtering and Rectification (as illustrated in + Figure 2), dCDN-2 will include the two Logging Records corresponding + respectively to the delivery performed by dCDN-3 and the delivery + performed by dCDN-2, in the next CDNI Logging File that will be + communicated to uCDN. An example of such CDNI Logging File is + illustrated below in Figure 6. + + #Version:CDNI/1.0 + + #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff + + #Claimed-Origin:cdni-logging-entity.dcdn-2.example.com + + #Record-Type:cdni_http_request_v1 + + #Fields:datetimetime-takenc-ip + cs-methodu-uriprotocolsc-status + sc-total-bytescs(User-Agent)cs(Referer) + s-cached + + 2013-05-1700:39:09.11914.0710.5.10.9GET + http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 + HTTP/1.120015799210"Mozilla/5.0 + (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like + Gecko) Chrome/5.0.375.127 Safari /533.4" + "host1.example.com"1 + + 2013-05-1701:42:53.43752.87910.5.10.12GET + http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 + HTTP/1.020097234724"Mozilla/5.0 + (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like + Gecko) Chrome/5.0.375.127 Safari /533.4" + "host5.example.com"0 + + #Integrity-Hash:fe113dfce8fec91323a4fc02261af26e + + [Editor's Note: compute the correct Integrity-Hash value once example + is frozen] + + Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) + + If uCDN establishes by some means (e.g. via TLS authentication when + pulling the CDNI Logging File) the identity of the entity from which + it pulled the CDNI Logging File, uCDN can add to the CDNI Logging a + Verified-Origin directive as illustrated below: + + #Verified-Origin:cdni-logging-entity.dcdn-2.example.com + + In the example of Figure 6, we observe that: + + o the first Logging Record corresponds to the Logging Record + communicated earlier to dCDN-2 by dCDN-3, which corresponds to a + delivery redirected by uCDN to dCDN-2 and then redirected by + dCDN-2 to dCDN-3. The fields values in this Logging Record are + copied from the corresponding CDNI Logging REcord communicated to + dCDN2 by dCDN-3, with the exception of the u-uri that now reflects + the URI convention between uCDN and dCDN-2 and that presents the + delivery to uCDN as if it was performed by dCDN-2 itself. This + reflects the fact that dCDN-2 had taken the full responsibility of + the corresponding delivery (even if in this case, dCDN-2 elected + to redirect the delivery to dCDN-3 so it is actually performed by + dCDN-3 on behalf of dCDN-2). + + o the second Logging Record corresponds to a delivery redirected by + uCDN to dCDN-2 and performed by dCDN-2 itself. The time of the + delivery in this Logging Record may be significantly more recent + than the first Logging Record since it was generated locally while + the first Logging Record was generated by dCDN-3 and had to be + advertised , and then pulled and then ingested into the dCDN-2 + Collection process, before being aggregated with the second + Logging Record. + +4. Protocol for Exchange of CDNI Logging File After Full Collection + + This section specifies a protocol for the exchange of CDNI Logging + Files as specified in Section 3 after the CDNI Logging File is fully + collected by the dCDN. This protocol comprises: o a CDNI Logging feed, allowing the dCDN to notify the uCDN about the CDNI Logging Files that can be retrieved by that uCDN from the dCDN, as well as all the information necessary for retrieving each of these CDNI Logging Files. The CDNI Logging feed is specified in Section 4.1. o a CDNI Logging File pull mechanism, allowing the uCDN to obtain from the dCDN a given CDNI Logging File at the uCDN's convenience. The CDNI Logging File pull mechanisms is specified in Section 4.2. An implementation of the CDNI Logging interface on the dCDN side (the entity generating the CDNI Logging file) MUST support the server side - of the CDNI Logging feed and the server side of the CDNI Logging pull - mechanism. + of the CDNI Logging feed (as specified in Section 4.1) and the server + side of the CDNI Logging pull mechanism (as specified in + Section 4.2). An implementation of the CDNI Logging interface on the uCDN side (the entity consuming the CDNI Logging file) MUST support the client side - of the CDNI Logging feed and the client side of the CDNI Logging pull - mechanism. - - We note that implementations of the CDNI Logging interface MAY also - support other mechanisms to exchange CDNI Logging Files, for example - in view of exchanging logging information with minimum time-lag - (e.g., sub-minute or sub-second) between when the event occurred in - the dCDN and when the corresponding Logging Record is made available - to the uCDN (e.g., for log-consuming applications requiring extremely - fresh logging information such as near-real-time content delivery - monitoring). Such mechanisms are for further study and outside the - scope of this document. + of the CDNI Logging feed (as specified in Section 4.1) and the client + side of the CDNI Logging pull mechanism (as specified in + Section 4.2). 4.1. CDNI Logging Feed The server-side implementation of the CDNI Logging feed MUST produce an Atom feed [RFC4287]. This feed is used to advertise log files that are available for the client-side to retrieve using the CDNI Logging pull mechanism. 4.1.1. Atom Formatting @@ -1487,75 +1676,84 @@ about older CDNI Logging files is moved into archive documents. Once created, archive documents are never modified. Each CDNI Logging File listed in an Atom feed MUST be described in an atom:entry container element. The atom:entry MUST contain an atom:content element whose "src" attribute is a link to the CDNI Logging File and whose "type" attribute is the MIME Media Type indicating that the entry is a CDNI Logging File. We define this MIME Media Type as "application/ - cdni.LoggingFile" (See Section 5.4). + cdni.LoggingFile" (See Section 6.4). For compatibility with some Atom feed readers the atom:entry MAY also contain an atom:link entry whose "href" attribute is a link to the CDNI Logging File and whose "type" attribute is the MIME Media Type indicating that the entry is a CDNI Logging File using the - "application/cdni.LoggingFile" MIME Media Type (See Section 5.4). + "application/cdni.LoggingFile" MIME Media Type (See Section 6.4). The URI used in the atom:id of the atom:entry MUST contain the UUID of the CDNI Logging File. The atom:updated in the atom:entry MUST indicate the time at which the CDNI Logging File was last updated. 4.1.2. Updates to Log Files and the Feed CDNI Logging Files MUST NOT be modified by the dCDN once published in the CDNI Logging feed. The frequency with which the subscription feed is updated, the period of time covered by each CDNI Logging File or each archive document, and timeliness of publishing of CDNI Logging Files are outside the scope of the present document and are expected to be agreed upon by uCDN and dCDN via other means (e.g., human agreement). - The server-side implementation SHOULD use HTTP cache control headers - on the subscription feed to indicate the frequency at which the - client-side is to poll for updates. + The server-side implementation MUST be able to set, and SHOULD set, + HTTP cache control headers on the subscription feed to indicate the + frequency at which the client-side is to poll for updates. + + The client-side MAY use HTTP cache control headers (set by the + server-side) on the subscription feed to determine the frequency at + which to poll for updates. The client-side MAY instead, or in + addition, use other information to determine when to poll for updates + (e.g., a polling frequency that may have been negotiated between the + uCDN and dCDN by mechanisms outside the scope of the present document + and that is to override the indications provided in the HTTP cache + control headers). The potential retention limits (e.g., sliding time window) within which the dCDN is to retain and be ready to serve an archive document is outside the scope of the present document and is expected to be agreed upon by uCDN and dCDN via other means (e.g., human agreement). The server-side implementation MUST retain, and be ready to serve, any archive document within the agreed retention limits. Outside these agreed limits, the server-side implementation MAY indicate its inability to serve (e.g., with HTTP status code 404) an archive document or MAY refuse to serve it (e.g., with HTTP status code 403 or 410). 4.1.3. Redundant Feeds The server-side implementation MAY present more than one CDNI Logging - feed and for redundancy. CDNI Logging Files MAY be published in more + feed for redundancy. Each CDNI Logging File MAY be published in more than one feed. A client-side implementation MAY support such redundant CDNI Logging feeds. If it supports redundant CDNI Logging feed, the client-side - SHOULD use the UUID of the CDNI Logging File, presented in the - atom:id element of the Atom feed, to avoid unnecessarily pulling and - storing each CDNI Logging File more than once. + can use the UUID of the CDNI Logging File, presented in the atom:id + element of the Atom feed, to avoid unnecessarily pulling and storing + a given CDNI Logging File more than once. 4.1.4. Example CDNI Logging Feed - Figure 4 illustrates an example of the subscription document of a + Figure 7 illustrates an example of the subscription document of a CDNI Logging feed. > CDNI Logging Feed 2013-03-23T14:46:11Z urn:uuid:663ae677-40fb-e99a-049d-c5642916b8ce @@ -1587,62 +1785,64 @@ type="application/cdni.LoggingFile" /> CDNI Logging File for uCDN at 2013-03-23 14:30:00 ... ... - Figure 4: Example subscription document of a CDNI Logging Feed + Figure 7: Example subscription document of a CDNI Logging Feed 4.2. CDNI Logging File Pull A client-side implementation of the CDNI Logging interface MAY pull, at its convenience, a CDNI Logging File that is published by the server-side in the CDNI Logging Feed (in the subscription document or an archive document). To do so, the client-side: - o MUST use HTTP v1.1 [RFC2616]; - - o SHOULD use TLS (i.e., use what is loosely referred to as "HTTPS") - as per [RFC2818] whenever protection of the CDNI Logging - information is required (see Section 6.1); + o MUST implement HTTP/1.1 [RFC2616], MAY also support other HTTP + versions (e.g., HTTP/2.0 [I-D.ietf-httpbis-http2]) and MAY + negotiate which HTTP version is actually used. This allows + operators and implementers to choose to use later versions of HTTP + to take advantage of new features, while still ensuring + interoperability with systems that only support HTTP/1.1. o MUST use the URI that was associated to the CDNI Logging File (within the "src" attribute of the corresponding atom:content element) in the CDNI Logging Feed; o MUST support exchange of CDNI Logging Files with no content encoding applied to the representation; - o SHOULD support exchange of CDNI Logging Files with "gzip" content + o MUST support exchange of CDNI Logging Files with "gzip" content encoding (as defined in [RFC2616]) applied to the representation. Note that a client-side implementation of the CDNI Logging interface MAY pull a CDNI Logging File that it has already pulled. The server-side implementation MUST respond to valid pull request by a client-side implementation for a CDNI Logging File published by the server-side in the CDNI Logging Feed (in the subscription document or an archive document). The server-side implementation: - o MUST handle the client-side request as per HTTP v1.1; + o MUST implement HTTP/1.1 to handle the client-side request and MAY + also support other HTTP versions (e.g., HTTP/2.0); o MUST include the CDNI Logging File identified by the request URI inside the body of the HTTP response; o MUST support exchange of CDNI Logging Files with no content encoding applied to the representation; - o SHOULD support exchange of CDNI Logging Files with "gzip" content + o MUST support exchange of CDNI Logging Files with "gzip" content encoding (as defined in [RFC2616]) applied to the representation. Content negotiation approaches defined in [RFC2616] (e.g., using Accept-Encoding request-header field or Content-Encoding entity- header field) MAY be used by the client-side and server-side implementations to establish the content-coding to be used for a particular exchange of a CDNI Logging File. Applying compression content encoding (such as "gzip") is expected to mitigate the impact of exchanging the large volumes of logging @@ -1657,47 +1857,70 @@ be ready to serve a CDNI Logging File previously advertised in the CDNI Logging Feed is outside the scope of the present document and is expected to be agreed upon by uCDN and dCDN via other means (e.g., human agreement). The server-side implementation MUST retain, and be ready to serve, any CDNI Logging File within the agreed retention limits. Outside these agreed limits, the server-side implementation MAY indicate its inability to serve (e.g., with HTTP status code 404) a CDNI Logging File or MAY refuse to serve it (e.g., with HTTP status code 403 or 410). -5. IANA Considerations +5. Protocol for Exchange of CDNI Logging File During Collection -5.1. CDNI Logging Directive Names Registry + We note that, in addition to the CDNI Logging File exchange protocol + specified in Section 4, implementations of the CDNI Logging interface + MAY also support other mechanisms to exchange CDNI Logging Files. In + particular, such mechanisms might allow the exchange of the CDNI + Logging File to start before the file is fully collected. This can + allow CDNI Logging Records to be communicated by the dCDN to the uCDN + as they are gathered by the dCDN without having to wait until all the + CDNI Logging Records of the same logging period are collected in the + corresponding CDNI Logging File. This approach is commonly referred + to as "tailing" of the file. + + Such an approach could be used, for example, to exchange logging + information with a significantly reduced time-lag (e.g., sub-minute + or sub-second) between when the event occurred in the dCDN and when + the corresponding CDNI Logging Record is made available to the uCDN. + This can satisfy log-consuming applications requiring extremely fresh + logging information such as near-real-time content delivery + monitoring. Such mechanisms are for further study and outside the + scope of this document. + +6. IANA Considerations + +6.1. CDNI Logging Directive Names Registry The IANA is requested to create a new registry, CDNI Logging Directive Names. The initial contents of the CDNI Logging File Directives registry comprise the names of the directives specified in Section 3.3 of the present document, and are as follows: +------------------------------+-----------+ - | Directive Name + Reference | + | Directive Name | Reference | +------------------------------+-----------+ - | Version + RFC xxxx | - | UUID + RFC xxxx | - | Claimed-Origin + RFC xxxx | - | Verified-Origin + RFC xxxx | - | Record-Type + RFC xxxx | - | Fields + RFC xxxx | - | Integrity-Hash + RFC xxxx | + | Version | RFC xxxx | + | UUID | RFC xxxx | + | Claimed-Origin | RFC xxxx | + | Verified-Origin | RFC xxxx | + | Record-Type | RFC xxxx | + | Fields | RFC xxxx | + | Integrity-Hash | RFC xxxx | +------------------------------+-----------+ - Figure 5 + Figure 8 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of the present document] + Within the registry, names are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Directive names are to be allocated by IANA with a format of NAMEFORMAT (see Section 3.1). Each specification that defines a new CDNI Logging directive needs to contain a description for the new directive with the same set of information as provided in Section 3.3 (i.e., format, directive value and occurrence). @@ -1694,191 +1917,200 @@ Within the registry, names are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Directive names are to be allocated by IANA with a format of NAMEFORMAT (see Section 3.1). Each specification that defines a new CDNI Logging directive needs to contain a description for the new directive with the same set of information as provided in Section 3.3 (i.e., format, directive value and occurrence). -5.2. CDNI Logging Record-Types Registry +6.2. CDNI Logging Record-Types Registry The IANA is requested to create a new registry, CDNI Logging Record- Types. The initial contents of the CDNI Logging Record-Types registry comprise the names of the CDNI Logging Record types specified in Section 3.4 of the present document, and are as follows: - +------------------------------+-----------+ - | Record-Types + Reference | - +------------------------------+-----------+ - | cdni_http_request_v1 + RFC xxxx | - +------------------------------+-----------+ + +------------------------------+-----------+----------------------------------+ + | Record-Types | Reference | Description | + +------------------------------+-----------+----------------------------------+ + | cdni_http_request_v1 | RFC xxxx | CDNI Logging Record version 1 | + | | | for content delivery using HTTP | + +------------------------------+-----------+----------------------------------+ - Figure 6 + Figure 9 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of the present document] Within the registry, Record-Types are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Record-Types are to be allocated by IANA with a format of - NAMEFORMAT (see Section 3.1). + NAMEFORMAT (see Section 3.1). Record-Types corresponding to + specifications produced by the IETF CDNI Working Group are to be + allocated a name starting with "cdni_". All other Record-Types are + to be allocated a name that does not start with "cdni". Each specification that defines a new Record-Type needs to contain a description for the new Record-Type with the same set of information as provided in Section 3.4.1. This includes: o a list of all the CDNI Logging Fields that can appear in a CDNI Logging Record of the new Record-Type o for all these Fields: a specification of the occurrence for each Field in the new Record-Type o for every newly defined Field, i.e., for every Field that results in a registration in the CDNI Logging Field Names Registry - (Section 5.3): a specification of the field name, format and field + (Section 6.3): a specification of the field name, format and field value. -5.3. CDNI Logging Field Names Registry +6.3. CDNI Logging Field Names Registry The IANA is requested to create a new registry, CDNI Logging Field Names. + The above registry is intended to be shared across the currently + defined Record-Type (i.e., cdni_http_request_v1) as well as potential + other CDNI Logging Record-Types that may be defined in separate + specifications. When a Field from this registry is used by another + CDNI Logging Record-Type, it is to be used with the exact semantics + and format specified in the document that registered this field and + that is identified in the Reference column of the registry. If + another CDNI Logging Record-Type requires a Field with semantics that + are not strictly identical, or a format that is not strictly + identical then this new Field is to be registered in the registry + with a different Field name. When a Field from this registry is used + by another CDNI Logging Record-Type, it can be used with different + occurence rules. + The initial contents of the CDNI Logging Fields Names registry comprise the names of the CDNI Logging fields specified in Section 3.4 of the present document, and are as follows: +---------------------------------------------+-----------+ - | Field Name + Reference | + | Field Name | Reference | +---------------------------------------------+-----------+ - | date + RFC xxxx | - | time + RFC xxxx | - | time-taken + RFC xxxx | - | c-ip + RFC xxxx | - | c-ip-anonymizing + RFC xxxx | - | c-port + RFC xxxx | - | s-ip + RFC xxxx | - | s-hostname + RFC xxxx | - | s-port + RFC xxxx | - | cs-method + RFC xxxx | - | cs-uri + RFC xxxx | - | u-uri + RFC xxxx | - | protocol + RFC xxxx | - | sc-status + RFC xxxx | - | sc-total-bytes + RFC xxxx | - | sc-entity-bytes + RFC xxxx | - | cs() + RFC xxxx | - | sc() + RFC xxxx | - | s-ccid + RFC xxxx | - | s-sid + RFC xxxx | - | s-cached + RFC xxxx | + | date | RFC xxxx | + | time | RFC xxxx | + | time-taken | RFC xxxx | + | c-ip | RFC xxxx | + | c-ip-anonymizing | RFC xxxx | + | c-port | RFC xxxx | + | s-ip | RFC xxxx | + | s-hostname | RFC xxxx | + | s-port | RFC xxxx | + | cs-method | RFC xxxx | + | cs-uri | RFC xxxx | + | u-uri | RFC xxxx | + | protocol | RFC xxxx | + | sc-status | RFC xxxx | + | sc-total-bytes | RFC xxxx | + | sc-entity-bytes | RFC xxxx | + | cs() | RFC xxxx | + | sc() | RFC xxxx | + | s-ccid | RFC xxxx | + | s-sid | RFC xxxx | + | s-cached | RFC xxxx | +---------------------------------------------+-----------+ - Figure 7 + Figure 10 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of the present document] Within the registry, names are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Field names are to be allocated by IANA with a format of NHTABSTRING (see Section 3.1). - The above registry is intended to be shared across the currently - defined Record-Type (i.e., cdni_http_request_v1) as well as potential - other CDNI Logging Record-Types that may be defined in separate - specifications. When a Field from this registry is used by another - CDNI Logging Record-Type, it is to be used with the exact semantics - and format specified in the document that registered this field and - that is identified in the Reference column of the registry. If - another CDNI Logging Record-Type requires a Field with semantics that - are not strictly identical, or a format that is not strictly - identical then this new Field is to be registered in the registry - with a different Field name. When a Field from this registry is used - by another CDNI Logging Record-Type, it can be used with different - occurence rules. - -5.4. CDNI Logging MIME Media Type +6.4. CDNI Logging MIME Media Type The IANA is requested to allocate the "application/cdni.LoggingFile" MIME Media Type (whose use is specified in Section 4.1.1 of the present document) in the MIME Media Types registry. -6. Security Considerations +7. Security Considerations -6.1. Authentication, Confidentiality, Integrity Protection +7.1. Authentication, Confidentiality, Integrity Protection - A CDNI Logging implementation MUST support TLS transport of the CDNI - Logging feed (Section 4.1) and of the CDNI Logging File pull - (Section 4.2) as per [RFC2818]. + An implementation of the CDNI Logging interface MUST support TLS + transport of the CDNI Logging feed (Section 4.1) and of the CDNI + Logging File pull (Section 4.2) as per [RFC2818]. The use of TLS for transport of the CDNI Logging feed and CDNI Logging File pull allows: o the dCDN and uCDN to authenticate each other (to ensure they are transmitting/receiving CDNI Logging File from an authenticated CDN) o the CDNI Logging information to be transmitted with confidentiality o the integrity of the CDNI Logging information to be protected during the exchange In an environment where any such protection is required, TLS SHOULD - be used for transport of the CDNI Logging feed and the CDNI Logging - File pull unless alternate methods are used for ensuring the - confidentiality of the information in the logging files (such as - setting up an IPsec tunnel between the two CDNs or using a physically - secured internal network between two CDNs that are owned by the same - corporate entity). Both parties of the transaction (uCDN and dCDN) - SHOULD use mutual authentication. + be used (including authentication of the remote end) by the server- + side and the client-side of the CDNI Logging feed and the CDNI + Logging File pull mechanism unless alternate methods are used for + ensuring the confidentiality of the information in the logging files + (such as setting up an IPsec tunnel between the two CDNs or using a + physically secured internal network between two CDNs that are owned + by the same corporate entity). - A CDNI Logging implementation MUST support the + An implementation of the CDNI Logging interface MUST support the TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite ( [RFC5288]). An - implementation of the CDNI Logging Interface SHOULD prefer cipher + implementation of the CDNI Logging interface SHOULD prefer cipher suites which support perfect forward secrecy over cipher suites that don't. The Integrity-Hash directive inside the CDNI Logging File provides additional integrity protection, this time targeting potential corruption of the CDNI logging information during the CDNI Logging - File generation. This mechanism does not allow restoration of the - corrupted CDNI Logging information, but it allows detection of such - corruption and therefore triggering of appropriate correcting actions - (e.g., discard of corrupted information, attempt to re-obtain the - CDNI Logging information). + File generation, storage or exchange. This mechanism does not itself + allow restoration of the corrupted CDNI Logging information, but it + allows detection of such corruption and therefore triggering of + appropriate corrective actions (e.g., discard of corrupted + information, attempt to re-obtain the CDNI Logging information). + Note that the Integrity-Hash does not protect against tampering by a + third party, since such a third party could have recomputed and + updated the Integrity-Hash after tampering. Protection against third + party tampering can be achieved as discussed above through the use of + TLS. -6.2. Denial of Service +7.2. Denial of Service This document does not define specific mechanism to protect against Denial of Service (DoS) attacks on the Logging Interface. However, the CDNI Logging feed and CDNI Logging pull endpoints can be protected against DoS attacks through the use of TLS transport and/or via mechanisms outside the scope of the CDNI Logging interface such as firewalling or use of Virtual Private Networks (VPNs). Protection of dCDN Surrogates against spoofed delivery requests is outside the scope of the CDNI Logging interface. -6.3. Privacy +7.3. Privacy CDNs have the opportunity to collect detailed information about the downloads performed by End Users. The provision of this information to another CDN introduces potential End Users privacy protection concerns. The use of TLS for transport of the CDNI Logging feed and CDNI - Logging pull as discussed in Section 6.1 protects the confidentiality + Logging pull as discussed in Section 7.1 protects the confidentiality of logged information by preventing any other party than the authorised uCDN to gain access to the logging information. We observe that when CDNI interconnection is realised as per [I-D.ietf-cdni-framework], the uCDN handles the initial End User requests (before it is redirected to the dCDN) so, regardless of which information is, or is not, communicated to the uCDN through the CDNI Logging interface, the uCDN has visibility on significant information such as the IP address of the End User request and the URL of the request. @@ -1905,38 +2137,38 @@ fields of CDNI Logging Files, or the HTTP cookies( [RFC6265]) that may be conveyed inside the cs() field of CDNI Logging Fields, may contain personnal information or information that can be exploited to derive personal information. Where this is a concern, the CDNI Logging interface specification allows the dCDN to not include the cs-uri and to include a u-uri that removes (or hides) the sensitive part of the query string and allows the dCDN to not include the cs() fields corresponding to HTTP headers associated with cookies. -7. Acknowledgments +8. Acknowledgments This document borrows from the W3C Extended Log Format [ELF]. Rob Murray significantly contributed into the text of Section 4.1. The authors thank Ben Niven-Jenkins, Kevin Ma, David Mandelberg and Ray van Brandenburg for their ongoing input. Finally, we also thank Sebastien Cubaud, Pawel Grochocki, Christian Jacquenet, Yannick Le Louedec, Anne Marrec , Emile Stephan, Fabio Costa, Sara Oueslati, Yvan Massot, Renaud Edel, Joel Favier and the contributors of the EU FP7 OCEAN project for their input in the early versions of this document. -8. References +9. References -8.1. Normative References +9.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC @@ -1956,49 +2188,50 @@ IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008. [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. [RFC5288] Salowey, J., Choudhury, A., and D. McGrew, "AES Galois Counter Mode (GCM) Cipher Suites for TLS", RFC 5288, August 2008. -8.2. Informative References +9.2. Informative References [CHAR_SET] - "IANA Character Sets registry", . + "IANA Character Sets registry", + . [ELF] Phillip M. Hallam-Baker, and Brian Behlendorf, "Extended - Log File Format, W3C (work in progress), WD- - logfile-960323", . + Log File Format, W3C (work in progress), WD-logfile- + 960323", . [I-D.ietf-cdni-framework] Peterson, L., Davie, B., and R. Brandenburg, "Framework - for CDN Interconnection", draft-ietf-cdni-framework-10 - (work in progress), March 2014. + for CDN Interconnection", draft-ietf-cdni-framework-14 + (work in progress), June 2014. [I-D.ietf-cdni-metadata] - Niven-Jenkins, B., Murray, R., Watson, G., Caulfield, M., - Leung, K., and K. Ma, "CDN Interconnect Metadata", draft- - ietf-cdni-metadata-06 (work in progress), February 2014. + Niven-Jenkins, B., Murray, R., Caulfield, M., Leung, K., + and K. Ma, "CDN Interconnection Metadata", draft-ietf- + cdni-metadata-07 (work in progress), July 2014. [I-D.ietf-cdni-requirements] Leung, K. and Y. Lee, "Content Distribution Network Interconnection (CDNI) Requirements", draft-ietf-cdni- requirements-17 (work in progress), January 2014. [I-D.ietf-httpbis-http2] Belshe, M., Peon, R., and M. Thomson, "Hypertext Transfer - Protocol version 2", draft-ietf-httpbis-http2-10 (work in - progress), February 2014. + Protocol version 2", draft-ietf-httpbis-http2-13 (work in + progress), June 2014. [I-D.snell-atompub-link-extensions] Snell, J., "Atom Link Extensions", draft-snell-atompub- link-extensions-09 (work in progress), June 2012. [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, April 1992. [RFC1945] Berners-Lee, T., Fielding, R., and H. Nielsen, "Hypertext Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996.