--- 1/draft-ietf-cdni-logging-16.txt 2015-03-19 09:14:38.279318310 -0700 +++ 2/draft-ietf-cdni-logging-17.txt 2015-03-19 09:14:38.379320740 -0700 @@ -1,22 +1,22 @@ Internet Engineering Task Force F. Le Faucheur, Ed. Internet-Draft Cisco Systems Intended status: Standards Track G. Bertrand, Ed. -Expires: September 10, 2015 I. Oprescu, Ed. +Expires: September 19, 2015 I. Oprescu, Ed. Orange R. Peterkofsky Skytide, Inc. - March 9, 2015 + March 18, 2015 CDNI Logging Interface - draft-ietf-cdni-logging-16 + draft-ietf-cdni-logging-17 Abstract This memo specifies the Logging interface between a downstream CDN (dCDN) and an upstream CDN (uCDN) that are interconnected as per the CDN Interconnection (CDNI) framework. First, it describes a reference model for CDNI logging. Then, it specifies the CDNI Logging File format and the actual protocol for exchange of CDNI Logging Files. @@ -28,21 +28,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at http://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on September 10, 2015. + This Internet-Draft will expire on September 19, 2015. Copyright Notice Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -60,58 +60,58 @@ 2. CDNI Logging Reference Model . . . . . . . . . . . . . . . . 5 2.1. CDNI Logging interactions . . . . . . . . . . . . . . . . 5 2.2. Overall Logging Chain . . . . . . . . . . . . . . . . . . 8 2.2.1. Logging Generation and During-Generation Aggregation 9 2.2.2. Logging Collection . . . . . . . . . . . . . . . . . 10 2.2.3. Logging Filtering . . . . . . . . . . . . . . . . . . 10 2.2.4. Logging Rectification and Post-Generation Aggregation 11 2.2.5. Log-Consuming Applications . . . . . . . . . . . . . 12 2.2.5.1. Maintenance/Debugging . . . . . . . . . . . . . . 12 2.2.5.2. Accounting . . . . . . . . . . . . . . . . . . . 12 - 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 12 - 2.2.5.4. Security . . . . . . . . . . . . . . . . . . . . 13 - 2.2.5.5. Legal Logging Duties . . . . . . . . . . . . . . 13 - 2.2.5.6. Notions common to multiple Log Consuming + 2.2.5.3. Analytics and Reporting . . . . . . . . . . . . . 13 + 2.2.5.4. Content Protection . . . . . . . . . . . . . . . 13 + 2.2.5.5. Notions common to multiple Log Consuming Applications . . . . . . . . . . . . . . . . . . 13 3. CDNI Logging File . . . . . . . . . . . . . . . . . . . . . . 15 3.1. Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 15 - 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 16 + 3.2. CDNI Logging File Structure . . . . . . . . . . . . . . . 17 3.3. CDNI Logging Directives . . . . . . . . . . . . . . . . . 19 3.4. CDNI Logging Records . . . . . . . . . . . . . . . . . . 23 3.4.1. HTTP Request Logging Record . . . . . . . . . . . . . 24 3.5. CDNI Logging File Example . . . . . . . . . . . . . . . . 33 - 3.6. Cascaded CDNI Logging Files Example . . . . . . . . . . . 34 + 3.6. Cascaded CDNI Logging Files Example . . . . . . . . . . . 35 4. Protocol for Exchange of CDNI Logging File After Full - Collection . . . . . . . . . . . . . . . . . . . . . . . . . 37 - 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 38 - 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 38 - 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 38 - 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 39 - 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 39 - 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 41 - 5. Protocol for Exchange of CDNI Logging File During Collection 42 - 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 43 - 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 43 - 6.2. CDNI Logging File Version Registry . . . . . . . . . . . 43 - 6.3. CDNI Logging Record-Types Registry . . . . . . . . . . . 44 - 6.4. CDNI Logging Field Names Registry . . . . . . . . . . . . 45 - 6.5. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 46 - 7. Security Considerations . . . . . . . . . . . . . . . . . . . 46 + Collection . . . . . . . . . . . . . . . . . . . . . . . . . 38 + 4.1. CDNI Logging Feed . . . . . . . . . . . . . . . . . . . . 39 + 4.1.1. Atom Formatting . . . . . . . . . . . . . . . . . . . 39 + 4.1.2. Updates to Log Files and the Feed . . . . . . . . . . 39 + 4.1.3. Redundant Feeds . . . . . . . . . . . . . . . . . . . 40 + 4.1.4. Example CDNI Logging Feed . . . . . . . . . . . . . . 40 + 4.2. CDNI Logging File Pull . . . . . . . . . . . . . . . . . 42 + 5. Protocol for Exchange of CDNI Logging File During Collection 43 + 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 44 + 6.1. CDNI Logging Directive Names Registry . . . . . . . . . . 44 + 6.2. CDNI Logging File Version Registry . . . . . . . . . . . 44 + 6.3. CDNI Logging Record-Types Registry . . . . . . . . . . . 45 + 6.4. CDNI Logging Field Names Registry . . . . . . . . . . . . 46 + 6.5. CDNI Logging MIME Media Type . . . . . . . . . . . . . . 47 + + 7. Security Considerations . . . . . . . . . . . . . . . . . . . 47 7.1. Authentication, Authorization, Confidentiality, Integrity - Protection . . . . . . . . . . . . . . . . . . . . . . . 47 - 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 48 - 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 48 - 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 49 - 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 49 - 9.1. Normative References . . . . . . . . . . . . . . . . . . 49 - 9.2. Informative References . . . . . . . . . . . . . . . . . 50 - Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 52 + Protection . . . . . . . . . . . . . . . . . . . . . . . 48 + 7.2. Denial of Service . . . . . . . . . . . . . . . . . . . . 49 + 7.3. Privacy . . . . . . . . . . . . . . . . . . . . . . . . . 49 + 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 50 + 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 50 + 9.1. Normative References . . . . . . . . . . . . . . . . . . 50 + 9.2. Informative References . . . . . . . . . . . . . . . . . 51 + Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 53 1. Introduction This memo specifies the CDNI Logging interface between a downstream CDN (dCDN) and an upstream CDN (uCDN). First, it describes a reference model for CDNI logging. Then, it specifies the CDNI Logging File format and the actual protocol for exchange of CDNI Logging Files. The reader should be familiar with the following documents: @@ -409,21 +409,21 @@ the full Logging information. Note that such aggregation leads to an information loss, which may be problematic for some usages of the Logging information (e.g., debugging). [RFC6983] discusses logging for HTTP Adaptive Streaming (HAS). In accordance with the recommendations articulated there, it is expected that a surrogate will generate separate Logging information for delivery of each chunk of HAS content. This ensures that separate Logging information can then be provided to interconnected CDNs over the CDNI Logging interface. Still in line with the recommendations - of [RFC6983], the Logging information for per-chunck delivery may + of [RFC6983], the Logging information for per-chunk delivery may include some information (a Content Collection IDentifier and a Session IDentifier) intended to facilitate subsequent post-generation aggregation of per-chunk logs into per-session logs. Note that a CDN may also elect to generate aggregate per-session logs when performing HAS delivery, but this needs to be in addition to, and not instead of, the per-chunk delivery logs. We note that aggregate per-session logs for HAS delivery are for further study and outside the scope of this document. 2.2.2. Logging Collection @@ -520,20 +520,29 @@ period, that content delivery related to a specific service succeeds/ fails. Logging information enables the CDN providers to identify and troubleshoot performance degradations. In particular, Logging information enables tracking of traffic data (e.g., the amount of traffic that has been forwarded by a dCDN on behalf of an uCDN over a given period of time), which is particularly useful for CDN and network planning operations. + Some of these maintenance and debugging applications only require + aggregate logging information highly compatible with anonymization of + IP addresses (as supported by the present document and specified in + Section 3.4.1). However, in some situations, it may be useful, where + compatible with privacy protection, to access some CDNI Logging + Records containing full non-anonymized IP addresses. For example, + this may be useful for detailed fault tracking of a particular end + user content delivery issue. + 2.2.5.2. Accounting Logging information is essential for accounting, to permit inter-CDN billing and CSP billing by uCDNs. For instance, Logging information provided by dCDNs enables the uCDN to compute the total amount of traffic delivered by every dCDN for a particular Content Provider, as well as, the associated bandwidth usage (e.g., peak, 95th percentile), and the maximum number of simultaneous sessions over a given period of time. @@ -544,54 +553,48 @@ quality of content delivery. For instance, Logging information enables the CDN providers to report on content consumption (e.g., delivered sessions per content) in a specific geographic area. The goal of reporting is to gather any relevant information to monitor the performance and quality of content delivery and allow detection of delivery issues. For instance, reporting could track the average delivery throughput experienced by End Users in a given region for a specific CSP or content set over a period of time. -2.2.5.4. Security - - The goal of security is to prevent and monitor unauthorized access, - misuse, modification, and denial of access of a service. A set of - information is logged for security purposes. In particular, a record - of access to content is usually collected to permit the CSP to detect - infringements of content delivery policies and other abnormal End - User behaviors. - -2.2.5.5. Legal Logging Duties +2.2.5.4. Content Protection - Depending on the country considered, the CDNs may have to retain - specific Logging information during a legal retention period, to - comply with judicial requisitions. + The goal of content protection is to prevent and monitor unauthorized + access, misuse, modification, and denial of access to a content. A + set of information is logged in a CDN for security purposes. In + particular, a record of access to content is usually collected to + permit the CSP to detect infringements of content delivery policies + and other abnormal End User behaviors. -2.2.5.6. Notions common to multiple Log Consuming Applications +2.2.5.5. Notions common to multiple Log Consuming Applications -2.2.5.6.1. Logging Information Views +2.2.5.5.1. Logging Information Views Within a given log-consuming application, different views may be provided to different users depending on privacy, business, and scalability constraints. For example, an analytics tool run by the uCDN can provide one view to an uCDN operator that exploits all the Logging information available to the uCDN, while the tool may provide a different view to each CSP exploiting only the Logging information related to the content of the given CSP. As another example, maintenance and debugging tools may provide different views to different CDN operators, based on their operational role. -2.2.5.6.2. Key Performance Indicators (KPIs) +2.2.5.5.2. Key Performance Indicators (KPIs) This section presents, for explanatory purposes, a non-exhaustive list of Key Performance Indicators (KPIs) that can be extracted/ produced from logs. Multiple log-consuming applications, such as analytics, monitoring, and maintenance applications, often compute and track such KPIs. In a CDNI environment, depending on the situation, these KPIs may be computed by the uCDN or by the dCDN. But it is usually the uCDN that @@ -631,21 +634,22 @@ month) o Cache-hit and byte-hit ratios for requests received from End Users in a given region for each piece of content, during a given period of time (e.g., hour/day/week/month) o Top 10 most popularly requested contents (during a given day/week/ month) o Terminal type (mobile, PC, STB, if this information can be - acquired from the browser type header, for example). + acquired from the browser type inferred from the User Agent + string, for example). Additional KPIs can be computed from other sources of information than the Logging information, for instance, data collected by a content portal or by specific client-side application programming interfaces. Such KPIs are out of scope for the present document. The KPIs used depend strongly on the considered log-consuming application -- the CDN operator may be interested in different metrics than the CSP is. In particular, CDN operators are often interested in delivery and acquisition performance KPIs, information @@ -694,43 +697,41 @@ IPv6address = as specified in section 3.2.2 of [RFC3986]. The present document also defines the following additional rules: ADDRESS = IPv4address / IPv6address ALPHANUM = ALPHA / DIGIT DATE = 4DIGIT "-" 2DIGIT "-" 2DIGIT - Dates are recorded in the format YYYY-MM-DD where YYYY, MM and - DD stand for the numeric year, month and day respectively. All - dates are specified in Universal Time Coordinated (UTC). + Dates are encoded as "full-date" specified in [RFC3339]. DEC = 1*DIGIT ["." *DIGIT] NAMEFORMAT = ALPHANUM *(ALPHANUM / "_" / "-") - QSTRING = DQUOTE *NDQUOTE DQUOTE ; where + QSTRING = DQUOTE *NDQUOTE DQUOTE NDQUOTE = / 2DQUOTE ; whereby a - DQUOTE is conveyed inside a QSTRING unambiguously by repeating - it. + DQUOTE is conveyed inside a QSTRING unambiguously by repeating it. - NHTABSTRING = *NHTAB ; where + [Editor's Note: The definition of NDQUOTE is being discussed as + part of IESG review and needs editing] + + NHTABSTRING = *NHTAB NHTAB = TIME = 2DIGIT ":" 2DIGIT ":" 2DIGIT ["." *DIGIT] - Times are recorded in the form HH:MM:SS or HH:MM:SS.S where HH - is the hour in 24 hour format, MM is minutes and SS is seconds. - All times are specified in Universal Time Coordinated (UTC). + Times are encoded as "partial-time" specified in [RFC3339]. 3.2. CDNI Logging File Structure As defined in Section 1.1: a CDNI Logging Field is as an atomic logging information element, a CDNI Logging Record is a collection of CDNI Logging Fields containing all logging information corresponding to a single logging event, and a CDNI Logging File contains a collection of CDNI Logging Records. This structure is illustrated in Figure 3. The use of a file structure for transfer of CDNI Logging information is selected since this is the most common practise today @@ -787,83 +788,90 @@ comply with the present document. The W3C Extended Log File Format was used as a starting point, reused where possible and expanded when necessary. Using a format that resembles the W3C Extended Log File Format is intended to keep CDNI logging format close to the intra-CDN Logging information format commonly used in CDNs today, thereby minimizing systematic translation at CDN/CDNI boundary. A CDNI Logging File MUST contain a sequence of lines containing US- - ASCII characters [CHAR_SET] terminated by CRLF. + ASCII characters [CHAR_SET] terminated by CRLF. [Editor's Note: This + needs editing to explain explain how CRLF/HTAB inside a QSTRING does + not act as a separator. ] Each line of a CDNI Logging File MUST contain either a directive or a CDNI Logging Record. Directives record information about the CDNI Logging process itself. Lines containing directives MUST begin with the "#" character. Directives are specified in Section 3.3. Logging Records provide actual details of the logged event. Logging Records are specified in Section 3.4. - The CDNI File structure is defined by the following rules: + The CDNI Logging File ("CDNILOGFILE") structure is defined by the + following rules: DIRLINE = "#" directive CRLF DIRGROUP = 1*DIRLINE - RECLINE = CRLF + RECLINE = CDNILOGREC CRLF RECGROUP = *RECLINE - = 1* + CDNILOGFILE = 1*(DIRGROUP RECGROUP) + + See Section 3.4 for the definition of CDNILOGREC. 3.3. CDNI Logging Directives The CDNI Logging directives are defined by the following rules: directive = DIRNAME ":" HTAB DIRVAL + DIRNAME = NAMEFORMAT DIRNAME = DIRVAL = An implementation of the CDNI Logging interface MUST support all of the following directives, listed below by their directive name: o Version: - * format: "CDNI" "/" 1*DIGIT "." 1*DIGIT + * format: NHTABSTRING * directive value: indicates the version of the CDNI Logging File format. The entity transmitting a CDNI Logging File as per the present document MUST set the value to "CDNI/1.0". In the future, other versions of CDNI Logging File might be specified; those would use a value different to "CDNI/1.0" allowing the entity receiving the CDNI Logging File to identify the corresponding version. * occurrence: there MUST be one and only one instance of this directive per CDNI Logging File. It MUST be the first line of the CDNI Logging File. o UUID: * format: NHTABSTRING - * directive value: this a Universally Unique IDentifier (UUID) - from the UUID Uniform Resource Name (URN) namespace specified - in [RFC4122]) for the CDNI Logging File. + * directive value: this a Uniform Resource Name (URN) from the + Universally Unique IDentifier (UUID) URN namespace specified in + [RFC4122]). The UUID contained in the URN uniquely identifies + the CDNI Logging File. * occurrence: there MUST be one and only one instance of this directive per CDNI Logging File. o Claimed-Origin: * format: host * directive value: this contains the claimed identification of the entity transmitting the CDNI Logging File (e.g., the host @@ -907,144 +916,145 @@ registry (Section 6.3). For example this may be "cdni_http_request_v1" as specified in Section 3.4.1. * occurrence: there MUST be at least one instance of this directive per CDNI Logging File. The first instance of this directive MUST precede a Fields directive and MUST precede all CDNI Logging Records. o Fields: - * format: FIENAME * ; where FIENAME can take any + * format: FIENAME *(HTAB FIENAME) ; where FIENAME can take any CDNI Logging field name registered in the CDNI Logging Field Names registry (Section 6.4). * directive value: this lists the names of all the fields for which a value is to appear in the CDNI Logging Records that follow the instance of this directive (until another instance of this directive). The names of the fields, as well as their occurrences, MUST comply with the corresponding rules specified in the document referenced in the CDNI Logging Record-types registry (Section 6.3) for the corresponding CDNI Logging Record-Type. * occurrence: there MUST be at least one instance of this directive per Record-Type directive. The first instance of this directive for a given Record-Type MUST appear before any CDNI Logging Record for this Record-Type. One situation where more than one instance of the Fields directive can appear within a given CDNI Logging File, is when there is a change, in the middle of a fairly large logging period, in the agreement between the uCDN and the dCDN about the set of Fields that are - to be exchanged. The multiple occurences allow records with + to be exchanged. The multiple occurrences allow records with the old set of fields and records with the new set of fields to be carried inside the same Logging File. - o Integrity-Hash: + o SHA256-Hash: * format: 32HEXDIG * directive value: This directive permits the detection of a corrupted CDNI Logging File. This can be useful, for instance, if a problem occurs on the filesystem of the dCDN Logging system and leads to a truncation of a logging file. The valid - Integrity-Hash value is included in this directive by the - entity that transmits the CDNI Logging File. It is computed by - applying the MD5 ([RFC1321]) cryptographic hash function on the - CDNI Logging File, including all the directives and logging - records, up to the Integrity-Hash directive itself, excluding - the Integrity-Hash directive itself. The Integrity-Hash value - is represented as a US-ASCII encoded hexadecimal number, 32 - digits long (representing a 128 bit hash value). The entity - receiving the CDNI Logging File also computes in a similar way - the MD5 hash on the received CDNI Logging File and compares - this hash to the value of the Integrity-Hash directive. If the - two values are equal, then the received CDNI Logging File MUST + SHA256-Hash value is included in this directive by the entity + that transmits the CDNI Logging File. It MUST be computed by + applying the SHA-256 ([RFC6234]) cryptographic hash function on + the CDNI Logging File, including all the directives and logging + records, up to the SHA256-Hash directive itself, excluding the + SHA256-Hash directive itself. The SHA256-Hash value MUST be + represented as a US-ASCII encoded hexadecimal number, 64 digits + long (representing a 256 bit hash value). The entity receiving + the CDNI Logging File also computes in a similar way the + SHA-256 hash on the received CDNI Logging File and compares + this hash to the value of the SHA256-Hash directive. If the + two values are equal, then the received CDNI Logging File is to be considered non-corrupted. If the two values are different, - the received CDNI Logging File MUST be considered corrupted. + the received CDNI Logging File is to be considered corrupted. The behavior of the entity that received a corrupted CDNI Logging File is outside the scope of this specification; we note that the entity MAY attempt to pull again the same CDNI Logging File from the transmitting entity. If the entity receiving a non-corrupted CDNI Logging File adds an Established-Origin directive, it MUST then recompute and update - the Integrity-Hash directive so it also protects the added + the SHA256-Hash directive so it also protects the added Established-Origin directive. * occurrence: there MUST be zero or exactly one instance of this directive. There SHOULD be exactly one instance of this directive. One situation where that directive could be omitted is where integrity protection is already provided via another mechanism (for example if an integrity hash is associated to the CDNI Logging File out of band through the CDNI Logging Feed ( Section 4.1) leveraging ATOM extensions such as those proposed in [I-D.snell-atompub-link-extensions]. When present, - the Integrity-Hash field MUST be the last line of the CDNI - Logging File. + the SHA256-Hash field MUST be the last line of the CDNI Logging + File. An uCDN-side implementation of the CDNI Logging interface MUST reject a CDNI Logging File that does not comply with the occurences specified above for each and every directive. For example, an uCDN- side implementation of the CDNI Logging interface receiving a CDNI Logging file with zero occurence of the Version directive, or with - two occurences of the Integrity-hash, MUST reject this CDNI Logging + two occurences of the SHA256-Hash, MUST reject this CDNI Logging File. An entity receiving a CDNI Logging File with a value set to "CDNI/1.0" MUST process the CDNI Logging File as per the present document. An entity receiving a CDNI Logging File with a value set to a different value MUST process the CDNI Logging File as per the specification referenced in the CDNI Logging File Version registry (see Section 6.1) if the implementation supports this specification and MUST reject the CDNI Logging File otherwise. 3.4. CDNI Logging Records A CDNI Logging Record consists of a sequence of CDNI Logging Fields relating to that single CDNI Logging Record. CDNI Logging Fields MUST be separated by the "horizontal tabulation - (HTAB)" character. + (HTAB)" character. [Editor's Note: This needs editing to explain + explain how CRLF/HTAB inside a QSTRING does not act as a separator. + ] To facilitate readability, a prefix scheme is used for CDNI Logging field names in a similar way to the one used in W3C Extended Log File Format [ELF]. The semantics of the prefix in the present document is: - o c: refers to the User Agent that issues the request (corresponds + o "c-" refers to the User Agent that issues the request (corresponds to the "client" of W3C Extended Log Format) - o d: refers to the dCDN (relative to a given CDN acting as a uCDN) + o "d-" refers to the dCDN (relative to a given CDN acting as a uCDN) - o s: refers to the dCDN Surrogate that serves the request + o "s-" refers to the dCDN Surrogate that serves the request (corresponds to the "server" of W3C Extended Log Format) - o u: refers to the uCDN (relative to a given CDN acting as a dCDN) + o "u-" refers to the uCDN (relative to a given CDN acting as a dCDN) - o cs: refers to communication from the User Agent towards the dCDN + o "cs-" refers to communication from the User Agent towards the dCDN Surrogate - o sc: refers to communication from the dCDN Surrogate towards the + o "sc-" refers to communication from the dCDN Surrogate towards the User Agent An implementation of the CDNI Logging interface as per the present specification MUST support the CDNI HTTP Request Logging Record as specified in Section 3.4.1. - A CDNI Logging Record is defined by the following rules: + A CDNI Logging Record (CDNILOGREC) is defined by the following rules: - FIEVAL = + CDNILOGREC = FIEVAL *(HTAB FIEVAL) - = FIEVAL * ; where FIEVAL - contains the CDNI Logging field value corresponding to the CDNI + FIEVAL = 3.4.1. HTTP Request Logging Record This section defines the CDNI Logging Record of Record-Type "cdni_http_request_v1". It is applicable to content delivery performed by the dCDN using HTTP/1.0([RFC1945]), HTTP/1.1([RFC7230],[RFC7231], [RFC7232], [RFC7233], [RFC7234], [RFC7235]) or HTTPS ([RFC2818], [RFC7230]). We observe that, in the case of HTTPS delivery, there may be value in logging additional information specific to the operation of HTTP over TLS and we note @@ -1106,40 +1117,58 @@ * field value: the source IPv4 or IPv6 address (i.e., the "client" address) in the request received by the Surrogate. * occurrence: there MUST be one and only one instance of this field. o c-ip-anonymizing: * format: 1*DIGIT - * field value: the number of rightmost bits of the address in the - c-ip field that are zeroed-out in order to anonymize the + * field value: the number of rightmost bits of the IPv4 address + in the c-ip field that are zeroed-out in order to anonymize the logging record. The mechanism by which the two ends of the CDNI Logging interface agree on whether anonymization is to be supported and the number of bits that need to be zeroed-out for this purpose are outside the scope of the present document. + IPv4 addresses SHOULD be anonymized to /24 boundary (i.e., with + c-ip-anonymizing set to 8), and IPv6 addresses SHOULD be + anonymized to a /48 boundary (i.e., with c-ip-anonymizing set + to 80). * occurrence: there MUST be zero or exactly one instance of this field. o c-port: * format: 1*DIGIT * field value: the source TCP port (i.e., the "client" port) in the request received by the Surrogate. * occurrence: there MUST be zero or exactly one instance of this field. + o c-port-anonymizing: + + * format: 1*DIGIT + + * field value: the number of rightmost bits of the port in the + c-port field that are zeroed-out in order to anonymize the + logging record. The mechanism by which the two ends of the + CDNI Logging interface agree on whether anonymization is to be + supported and the number of bits that need to be zeroed-out for + this purpose are outside the scope of the present document. + + * occurrence: there MUST be zero or exactly one instance of this + field. + o s-ip: * format: ADDRESS * field value: the IPv4 or IPv6 address of the Surrogate that served the request (i.e., the "server" address). * occurrence: there MUST be zero or exactly one instance of this field. @@ -1173,21 +1203,23 @@ * occurrence: There MUST be one and only one instance of this field. o cs-uri: * format: NHTABSTRING * field value: this is the "effective request URI" of the request received by the Surrogate as specified in [RFC7230]. It complies with the "http" URI scheme or the "https" URI scheme - as specified in [RFC7230]). + as specified in [RFC7230]). Note that cs-uri can be privacy + sensitive. In that case, and where appropriate, u-uri could be + used instead of cs-uri. * occurrence: there MUST be zero or exactly one instance of this field. o u-uri: * format: NHTABSTRING * field value: this is a complete URI, derived from the "effective request URI" ([RFC7230]) of the request received by @@ -1394,27 +1425,32 @@ to include valid values for each of them: o date o time o time-taken o c-ip + o c-ip-anonymizing + o c-port + o c-port-anonymizing + o s-ip o s-hostname o s-port + o cs-method o cs-uri o u-uri o protocol o sc-status @@ -1425,22 +1460,20 @@ o cs() o sc() o s-cached A dCDN-side implementation of the CDNI Logging interface MAY support the following Logging Fields in a CDNI Logging Record of Record-Type "cdni_http_request_v1": - o c-ip-anonymizing - o s-ccid o s-sid If a dCDN-side implementation of the CDNI Logging interface supports these Fields, it MUST support the ability to include valid values for them. An uCDN-side implementation of the CDNI Logging interface MUST be able to accept CDNI Logging Files with CDNI Logging Records of @@ -1467,46 +1500,46 @@ #Version:CDNI/1.0 #UUID:urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6 #Claimed-Origin:cdni-logging-entity.dcdn-1.example.com #Record-Type:cdni_http_request_v1 #Fields:datetimetime-takenc-ip - cs-methodu-uriprotocolsc-status - sc-total-bytescs(User-Agent)cs(Referer) - s-cached +c-ip-anonymizingcs-methodu-uriprotocol +sc-statussc-total-bytescs(User-Agent) +cs(Referer)s-cached - 2013-05-1700:38:06.8259.05810.5.7.1GET +2013-05-1700:38:06.8259.05810.5.7.08GET http://cdni-ucdn.dcdn-1.example.com/video/movie100.mp4 HTTP/1.12006729891"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari/533.4" "host1.example.com"1 - 2013-05-1700:39:09.14515.3210.5.10.5GET +2013-05-1700:39:09.14515.3210.5.10.08GET http://cdni-ucdn.dcdn-1.example.com/video/movie118.mp4 HTTP/1.120015799210"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari/533.4" "host1.example.com"1 - 2013-05-1700:42:53.43752.87910.5.10.5GET +2013-05-1700:42:53.43752.87910.5.10.08GET http://cdni-ucdn.dcdn-1.example.com/video/picture11.mp4 HTTP/1.020097234724"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari/533.4" "host5.example.com"0 - #Integrity-Hash:...32-hexadecimal-digit hash value... +#SHA256-Hash:...32-hexadecimal-digit hash value... Figure 4: CDNI Logging File Example If uCDN establishes by some means (e.g. via TLS authentication when pulling the CDNI Logging File) the identity of the entity from which it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an Established-Origin directive as illustrated below: #Established-Origin:cdni-logging-entity.dcdn- 1.example.com @@ -1556,32 +1588,32 @@ #Version:CDNI/1.0 #UUID:urn:uuid:65718ef-0123-9876-adce4321bcde #Claimed-Origin:cdni-logging-entity.dcdn-3.example.com #Record-Type:cdni_http_request_v1 #Fields:datetimetime-takenc-ip - cs-methodu-uriprotocolsc-status - sc-total-bytescs(User-Agent)cs(Referer) - s-cached +c-ip-anonymizingcs-methodu-uriprotocol +sc-statussc-total-bytescs(User-Agent) +cs(Referer)s-cached - 2013-05-1700:39:09.11914.0710.5.10.9GET +2013-05-1700:39:09.11914.0710.5.10.08GET http://cdni-dcdn-2.dcdn-3.example.com/video/movie118.mp4 HTTP/1.120015799210"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari /533.4" "host1.example.com"1 - #Integrity-Hash:...32-hexadecimal-digit hash value... +#SHA256-Hash:...32-hexadecimal-digit hash value... Figure 5: Cascaded CDNI Logging File Example (dCDN-3 to dCDN-2) If dCDN-2 establishes by some means (e.g. via TLS authentication when pulling the CDNI Logging File) the identity of the entity from which it pulled the CDNI Logging File, dCDN-2 can add to the CDNI Logging an Established-Origin directive as illustrated below: #Established-Origin:cdni-logging-entity.dcdn- 3.example.com @@ -1604,39 +1636,39 @@ #Version:CDNI/1.0 #UUID:urn:uuid:1234567-8fedc-abab-0987654321ff #Claimed-Origin:cdni-logging-entity.dcdn-2.example.com #Record-Type:cdni_http_request_v1 #Fields:datetimetime-takenc-ip - cs-methodu-uriprotocolsc-status - sc-total-bytescs(User-Agent)cs(Referer) - s-cached +c-ip-anonymizingcs-methodu-uriprotocol +sc-statussc-total-bytescs(User-Agent) +cs(Referer)s-cached - 2013-05-1700:39:09.11914.0710.5.10.9GET +2013-05-1700:39:09.11914.0710.5.10.08GET http://cdni-ucdn.dcdn-2.example.com/video/movie118.mp4 HTTP/1.120015799210"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari /533.4" "host1.example.com"1 - 2013-05-1701:42:53.43752.87910.5.10.12GET +2013-05-1701:42:53.43752.87910.5.10.08GET http://cdni-ucdn.dcdn-2.example.com/video/picture11.mp4 HTTP/1.020097234724"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.127 Safari /533.4" "host5.example.com"0 - #Integrity-Hash:...32-hexadecimal-digit hash value... +#SHA256-Hash:...32-hexadecimal-digit hash value... Figure 6: Cascaded CDNI Logging File Example (dCDN-2 to uCDN) If uCDN establishes by some means (e.g. via TLS authentication when pulling the CDNI Logging File) the identity of the entity from which it pulled the CDNI Logging File, uCDN can add to the CDNI Logging an Established-Origin directive as illustrated below: #Established-Origin:cdni-logging-entity.dcdn- 2.example.com @@ -1934,21 +1966,21 @@ +------------------------------+-----------+ | Directive Name | Reference | +------------------------------+-----------+ | Version | RFC xxxx | | UUID | RFC xxxx | | Claimed-Origin | RFC xxxx | | Established-Origin | RFC xxxx | | Record-Type | RFC xxxx | | Fields | RFC xxxx | - | Integrity-Hash | RFC xxxx | + | SHA256-Hash | RFC xxxx | +------------------------------+-----------+ Figure 8 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of the present document] Within the registry, names are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Directive names are to be allocated by IANA with a format of @@ -1976,21 +2008,21 @@ +-----------------+-----------+----------------------------------+ Figure 9 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of the present document] Within the registry, Version values are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Version values are to be allocated by IANA with a format - as specified for the Version directive in Section 3.3. + of NAMEFORMAT (see Section 3.1). 6.3. CDNI Logging Record-Types Registry The IANA is requested to create a new registry, CDNI Logging Record- Types. The initial contents of the CDNI Logging Record-Types registry comprise the names of the CDNI Logging Record types specified in Section 3.4 of the present document, and are as follows: @@ -2002,24 +2034,21 @@ +----------------------+-----------+----------------------------------+ Figure 10 [Instructions to IANA: Replace "RFC xxxx" above by the RFC number of the present document] Within the registry, Record-Types are to be allocated by IANA according to the "Specification Required" policy specified in [RFC5226]. Record-Types are to be allocated by IANA with a format of - NAMEFORMAT (see Section 3.1). Record-Types corresponding to - specifications produced by the IETF CDNI Working Group are to be - allocated a name starting with "cdni_". All other Record-Types are - to be allocated a name that does not start with "cdni". + NAMEFORMAT (see Section 3.1). Each specification that defines a new Record-Type needs to contain a description for the new Record-Type with the same set of information as provided in Section 3.4.1. This includes: o a list of all the CDNI Logging Fields that can appear in a CDNI Logging Record of the new Record-Type o for all these Fields: a specification of the occurrence for each Field in the new Record-Type @@ -2114,43 +2143,39 @@ CDN) o the CDNI Logging information to be transmitted with confidentiality o the integrity of the CDNI Logging information to be protected during the exchange. In an environment where any such protection is required, the use of a mutually authenticated encrypted transport MUST be used to ensure - confidentiality of the logging information. TLS SHOULD be used + confidentiality of the logging information. TLS MUST be used (including authentication of the remote end) by the server- side and the client-side of the CDNI Logging feed, as well as the server- side - and the client-side of the CDNI Logging File pull mechanism, unless - alternate methods are used. Alternate methods could include - establishing an IPsec tunnel between the two CDNs or using a - physically secured internal network between two CDNs that are owned - by the same corporate entity). + and the client-side of the CDNI Logging File pull mechanism. The general TLS usage guidance in [I-D.ietf-uta-tls-bcp] SHOULD be followed. - The Integrity-Hash directive inside the CDNI Logging File provides + The SHA256-Hash directive inside the CDNI Logging File provides additional integrity protection, this time targeting potential corruption of the CDNI logging information during the CDNI Logging File generation, storage or exchange. This mechanism does not itself allow restoration of the corrupted CDNI Logging information, but it allows detection of such corruption and therefore triggering of appropriate corrective actions (e.g., discard of corrupted information, attempt to re-obtain the CDNI Logging information). - Note that the Integrity-Hash does not protect against tampering by a + Note that the SHA256-Hash does not protect against tampering by a third party, since such a third party could have recomputed and - updated the Integrity-Hash after tampering. Protection against third + updated the SHA256-Hash after tampering. Protection against third party tampering can be achieved as discussed above through the use of TLS. 7.2. Denial of Service This document does not define specific mechanism to protect against Denial of Service (DoS) attacks on the Logging Interface. However, the CDNI Logging feed and CDNI Logging pull endpoints are typically to be accessed only by a very small number of valid remote endpoints and therefore can be easily protected against DoS attacks through the @@ -2230,20 +2255,23 @@ 9.1. Normative References [I-D.ietf-uta-tls-bcp] Sheffer, Y., Holz, R., and P. Saint-Andre, "Recommendations for Secure Use of TLS and DTLS", draft- ietf-uta-tls-bcp-11 (work in progress), February 2015. [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. + [RFC3339] Klyne, G., Ed. and C. Newman, "Date and Time on the + Internet: Timestamps", RFC 3339, July 2002. + [RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, January 2005. [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally Unique IDentifier (UUID) URN Namespace", RFC 4122, July 2005. [RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom Syndication Format", RFC 4287, December 2005. @@ -2301,28 +2329,28 @@ [I-D.ietf-httpbis-http2] Belshe, M., Peon, R., and M. Thomson, "Hypertext Transfer Protocol version 2", draft-ietf-httpbis-http2-17 (work in progress), February 2015. [I-D.snell-atompub-link-extensions] Snell, J., "Atom Link Extensions", draft-snell-atompub- link-extensions-09 (work in progress), June 2012. - [RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm", RFC 1321, - April 1992. - [RFC1945] Berners-Lee, T., Fielding, R., and H. Nielsen, "Hypertext Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996. [RFC2818] Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000. + [RFC6234] Eastlake, D. and T. Hansen, "US Secure Hash Algorithms + (SHA and SHA-based HMAC and HKDF)", RFC 6234, May 2011. + [RFC6265] Barth, A., "HTTP State Management Mechanism", RFC 6265, April 2011. [RFC6707] Niven-Jenkins, B., Le Faucheur, F., and N. Bitar, "Content Distribution Network Interconnection (CDNI) Problem Statement", RFC 6707, September 2012. [RFC6770] Bertrand, G., Stephan, E., Burbridge, T., Eardley, P., Ma, K., and G. Watson, "Use Cases for Content Delivery Network Interconnection", RFC 6770, November 2012.