< draft-ietf-sipclf-format-04.txt   draft-ietf-sipclf-format-05.txt >
SIPCLF G. Salgueiro SIPCLF G. Salgueiro
Internet-Draft Cisco Systems Internet-Draft Cisco Systems
Intended status: Standards Track V. Gurbani Intended status: Standards Track V. Gurbani
Expires: June 9, 2012 Bell Labs, Alcatel-Lucent Expires: June 19, 2012 Bell Labs, Alcatel-Lucent
A. B. Roach A. B. Roach
Tekelec Tekelec
December 7, 2011 December 17, 2011
Format for the Session Initiation Protocol (SIP) Common Log Format (CLF) Format for the Session Initiation Protocol (SIP) Common Log Format (CLF)
draft-ietf-sipclf-format-04 draft-ietf-sipclf-format-05
Abstract Abstract
The SIPCLF Workgroup has defined a common log format framework for The SIPCLF Workgroup has defined a common log format framework for
Session Initiation Protocol (SIP) servers. This common log format Session Initiation Protocol (SIP) servers. This common log format
mimics the wildly successful event logging mechanism found in well- mimics the successful event logging format found in well-known web
known web servers like Apache and web proxies like Squid. This servers like Apache and web proxies like Squid. This document
document proposes an indexed text encoding format for the SIP Common proposes an indexed text encoding format for the SIP Common Log
Log Format (CLF) that retains the key advantages of a text-based Format (CLF) that retains the key advantages of a text-based format,
format, while significantly increasing processing performance over a while significantly increasing processing performance over a purely
purely text-based implementation. This file format adheres to the text-based implementation. This file format adheres to the SIP CLF
SIP CLF data model and provides an effective encoding scheme for all data model and provides an effective encoding scheme for all
mandatory and optional fields that appear in a SIP CLF record. mandatory and optional fields that appear in a SIP CLF record.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on June 9, 2012. This Internet-Draft will expire on June 19, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 23 skipping to change at page 2, line 23
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Document Conventions . . . . . . . . . . . . . . . . . . . . . 4 3. Document Conventions . . . . . . . . . . . . . . . . . . . . . 4
4. Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4. Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.1. Index Pointers . . . . . . . . . . . . . . . . . . . . . . 9 4.1. Index Pointers . . . . . . . . . . . . . . . . . . . . . . 9
4.2. Mandatory Fields . . . . . . . . . . . . . . . . . . . . . 11 4.2. Mandatory Fields . . . . . . . . . . . . . . . . . . . . . 11
4.3. Optional Fields . . . . . . . . . . . . . . . . . . . . . 15 4.3. Optional Fields . . . . . . . . . . . . . . . . . . . . . 15
5. Example SIP CLF Record . . . . . . . . . . . . . . . . . . . . 22 5. Example SIP CLF Record . . . . . . . . . . . . . . . . . . . . 22
6. Text Tool Considerations . . . . . . . . . . . . . . . . . . . 23 6. Text Tool Considerations . . . . . . . . . . . . . . . . . . . 24
7. Security Considerations . . . . . . . . . . . . . . . . . . . 24 7. Security Considerations . . . . . . . . . . . . . . . . . . . 24
8. Operational Guidance . . . . . . . . . . . . . . . . . . . . . 24 8. Operational Guidance . . . . . . . . . . . . . . . . . . . . . 24
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 24 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 25
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 24 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 25
11.1. Normative References . . . . . . . . . . . . . . . . . . . 25 11.1. Normative References . . . . . . . . . . . . . . . . . . . 25
11.2. Informative References . . . . . . . . . . . . . . . . . . 25 11.2. Informative References . . . . . . . . . . . . . . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction 1. Introduction
The extensive list of benefits and the widespread adoption of the The extensive list of benefits and the widespread adoption of the
Apache Common Log Format (CLF) has prompted the development of a Apache Common Log Format (CLF) has prompted the development of a
functionally equivalent event logging mechanism for the Session functionally equivalent event logging mechanism for the Session
Initiation Protocol [RFC3261] (SIP). Implementing a logging scheme Initiation Protocol [RFC3261] (SIP). Implementing a logging scheme
for SIP is a considerable challenge. This is due in part to the fact for SIP is a considerable challenge. This is due in part to the fact
that the behavior of a SIP entity is more complex as compared to an that the behavior of a SIP entity is more complex as compared to an
HTTP entity. Additionally, there are shortcomings to the purely HTTP entity. Additionally, there are shortcomings to the purely
text-based HTTP Common Log Format that need to be addressed in order text-based HTTP Common Log Format that need to be addressed in order
to allow for real-time inspection of SIP log files. Experience with to allow for real-time inspection of SIP log files
Apache Common Log Format has shown that dealing with large quantities [I-D.ietf-sipclf-problem-statement]. Experience with Apache Common
of log data can be very processor intensive, as doing so necessarily Log Format has shown that dealing with large quantities of log data
requires reading and parsing every byte in the log file(s) of can be very processor intensive, as doing so necessarily requires
interest. reading and parsing every byte in the log file(s) of interest.
An implementation independent framework for the SIP CLF has been An implementation independent framework for the SIP CLF has been
defined in [I-D.ietf-sipclf-problem-statement]. This memo describes defined in [I-D.ietf-sipclf-problem-statement]. This memo describes
an indexed text file format for logging SIP messages received and an indexed text file format for logging SIP messages received and
sent by SIP clients, servers, and proxies that adheres to the data sent by SIP clients, servers, and proxies that adheres to the data
model presented in Section 8 of [I-D.ietf-sipclf-problem-statement]. model presented in Section 8 of [I-D.ietf-sipclf-problem-statement].
This document defines a format that is no more difficult to generate This document defines a format that is no more difficult to generate
by logging entities, while being radically faster to process. In by logging entities, while being radically faster to process. In
particular, the format is optimized for both rapidly scanning through particular, the format is optimized for both rapidly scanning through
log records, as well as quickly locating commonly accessed data log records, as well as quickly locating commonly accessed data
skipping to change at page 5, line 15 skipping to change at page 5, line 15
Header-name: first value, Header-name: first value,
reallylong reallylong
second second
value, value,
third value third value
</allOneLine> </allOneLine>
Note that this is NOT SIP header-line folding, where different Note that this is NOT SIP header-line folding, where different
strings of bits have equivalent meaning. strings of bits have equivalent meaning.
The ip addresses used in the examples in this document adhere to the The IP addresses used in the examples in this document adhere to the
best practices outlined in [RFC5735] and correspond to the best practices outlined in [RFC5735] and correspond to the
documentation address block 192.0.2.0/24 (TEST-NET-1) as described in documentation address block 192.0.2.0/24 (TEST-NET-1) as described in
[RFC5737]. [RFC5737].
4. Format 4. Format
The Common Log Format for the Session Initiation Protocol The Common Log Format for the Session Initiation Protocol
[I-D.ietf-sipclf-problem-statement] defines a data model to which [I-D.ietf-sipclf-problem-statement] defines a data model to which
this logging format format adheres. Each SIP CLF record MUST consist this logging format format adheres. Each SIP CLF record MUST consist
of all the mandatory data model elements outlined in Section 8.1 of of all the mandatory data model elements outlined in Section 8.1 of
skipping to change at page 7, line 14 skipping to change at page 7, line 14
The format presented in Figure 1 is for a single SIP CLF log entry. The format presented in Figure 1 is for a single SIP CLF log entry.
While there is no actual subdivision in practice, this format can be While there is no actual subdivision in practice, this format can be
logically subdivided into the following three distinct components: logically subdivided into the following three distinct components:
1. Index Pointers - The first 60-bytes of this format. This 1. Index Pointers - The first 60-bytes of this format. This
portion is metadata, primarily composed of a list of pointers that portion is metadata, primarily composed of a list of pointers that
indicate the beginning of both the variable length mandatory and indicate the beginning of both the variable length mandatory and
optional fields that are logged as part of this record. These optional fields that are logged as part of this record. These
pointers are implemented as a mechanism to improve processing of pointers are implemented as a mechanism to improve processing of
these records and to allow a reader to expeditiously skip right to these records and to allow a reader to expeditiously skip directly
the desired field without unnecessarily going through the entire to the desired field without unnecessarily going through the
record. This logical subdivision within the SIP CLF format will entire record. This logical subdivision within the SIP CLF format
be referenced in this document with the <IndexPointers> tag. will be referenced in this document with the <IndexPointers> tag.
A 0x0A (LF character) delimits <IndexPointers> from the next
logical grouping.
2. Mandatory Fields - The next logical grouping in this format is 2. Mandatory Fields - The next logical grouping in this format is
a tab delimited listing of the mandatory fields as described in a tab (0x09) delimited listing of the mandatory fields as
Section 8.1 of [I-D.ietf-sipclf-problem-statement] and in the described in Section 8.1 of [I-D.ietf-sipclf-problem-statement]
order listed in <IndexPointers>. This logical subdivision within and in the order listed in <IndexPointers>. This logical
the SIP CLF format will be referenced in this document with the subdivision within the SIP CLF format will be referenced in this
<MandatoryFields> tag. document with the <MandatoryFields> tag.
3. Optional Fields - The last logical component MAY be present as 3. Optional Fields - The last logical component MAY be present as
it is an OPTIONAL extension to the SIP CLF format. Its purpose is it is an OPTIONAL extension to the SIP CLF format. Its purpose is
to provide flexibility to the developer of this SIP CLF to log any to provide flexibility to the developer of this SIP CLF to log any
desired fields not included in <MandatoryFields>. This includes desired fields not included in <MandatoryFields>. This includes
SIP bodies and any vendor-specific extensions. This logical SIP bodies and any vendor-specific extensions. This logical
subdivision within the SIP CLF format will be referenced in this subdivision within the SIP CLF format will be referenced in this
document with the <OptionalFields> tag. document with the <OptionalFields> tag.
This logical structure of the SIP CLF record format can be This logical structure of the SIP CLF record format can be
graphically represented as shown in Figure 2 below: graphically represented as shown in Figure 2 below:
<IndexPointers> <IndexPointers>
<MandatoryFields> <MandatoryFields>
<OptionalFields> <OptionalFields>
Figure 2: Logical Structure of the SIP CLF Record Figure 2: Logical Structure of the SIP CLF Record
Note that Figure 1 and Figure 2 plus the terminating line-feed at the Note that Figure 1 and Figure 2 plus the terminating line-feed (0x0A)
end of the SIP CLF record are different representations of the same at the end of the SIP CLF record are different representations of the
format but are functionally equivalent. The representation of this same format but are functionally equivalent. The representation of
format is a two line record where the <IndexPointers> metadata is on this format is a two line record where the <IndexPointers> metadata
one line and the actual data like <MandatoryFields> and is on one line and the actual data like <MandatoryFields> and
<OptionalFields> (if present) is on another. <OptionalFields> (if present) is on another.
In the following sections note that indications of "hexadecimal In the following sections note that indications of "hexadecimal
encoded" indicate that the value is to be written out in human- encoded" indicate that the value is to be written out in human-
readable base-16 numbers using the ASCII characters 0x30 through 0x39 readable base-16 numbers using the ASCII characters 0x30 through 0x39
('0' through '9') and 0x41 through 0x46 ('A' through 'F'). ('0' through '9') and 0x41 through 0x46 ('A' through 'F').
Similarly, indications of "decimal encoded" indicate that the value Similarly, indications of "decimal encoded" indicate that the value
is to be written out in human readable base-10 number using the ASCII is to be written out in human readable base-10 number using the ASCII
characters 0x30 through 0x39 ('0' through '9'). In both encodings, characters 0x30 through 0x39 ('0' through '9'). In both encodings,
numbers always take up the number of bytes indicated, and are padded numbers always take up the number of bytes indicated, and are padded
skipping to change at page 14, line 21 skipping to change at page 14, line 21
[I-D.ietf-sipclf-problem-statement] for usage.) [I-D.ietf-sipclf-problem-statement] for usage.)
This data MUST appear in the order listed in <IndexPointers>, and This data MUST appear in the order listed in <IndexPointers>, and
each field MUST be present. Fields are subject the maximum SIP CLF each field MUST be present. Fields are subject the maximum SIP CLF
field size of 4096 bytes as detailed in Section 8 of field size of 4096 bytes as detailed in Section 8 of
[I-D.ietf-sipclf-problem-statement] and are separated by a single [I-D.ietf-sipclf-problem-statement] and are separated by a single
ASCII Tab character (0x09). Any Tab characters present in the data ASCII Tab character (0x09). Any Tab characters present in the data
to be written will be replaced by an ASCII space character (0x20) to be written will be replaced by an ASCII space character (0x20)
prior to being logged. prior to being logged.
Table 1 of Section 8.2 of [I-D.ietf-sipclf-problem-statement] An element will not always have an appropriate value to provide for
summarizes how the mandatory fields are logged by the various SIP one of these fields, even when the field is required to appear in the
entities. This illustrates the fact that there are instances when a SIP CLF record. In such circumstances, when a given mandatory field
given mandatory field is not applicable for logging in the SIP CLF is not present then that empty field MUST be encoded as a single
because it does not make sense based on the role the entity is horizontal dash ("-").
playing in the SIP ecosystem. In such circumstances, if a given
mandatory field is not present then that empty field MUST be encoded
as a single horizontal dash ("-").
In the event that a field failed to parse it MUST be encoded as a In the event that a field failed to parse it MUST be encoded as a
single question mark ("?"). If these characters are part of a single question mark ("?"). If these characters are part of a
sequence of other characters, then there is no ambiguity. If the sequence of other characters, then there is no ambiguity. If the
field being logged contains only one character, and that character is field being logged contains only one character, and that character is
the literal "-", the implementation SHOULD insert an escaped %2D for the literal "-", the implementation SHOULD insert an escaped %2D for
that field in the SIP CLF record. Similarly, if the field contains that field in the SIP CLF record. Similarly, if the field contains
only one character, and that character is the literal "?", the only one character, and that character is the literal "?", the
implementation SHOULD insert an escaped %3F for that field in the SIP implementation SHOULD insert an escaped %3F for that field in the SIP
CLF record. CLF record.
skipping to change at page 23, line 22 skipping to change at page 23, line 22
A0000FE,0053005C005E006D007D008F009E00A000BA00C700EB00F500FE A0000FE,0053005C005E006D007D008F009E00A000BA00C700EB00F500FE
<allOneLine> <allOneLine>
0000000000.010 RORUU 1 INVITE - sip:192.0.2.10 0000000000.010 RORUU 1 INVITE - sip:192.0.2.10
192.0.2.10:5060 192.0.2.200:56485 sip:192.0.2.10 - 192.0.2.10:5060 192.0.2.200:56485 sip:192.0.2.10 -
sip:1001@example.com:5060 DL88360fa5fc sip:1001@example.com:5060 DL88360fa5fc
DL70dff590c1-1079051554@example.com server-tx client-tx DL70dff590c1-1079051554@example.com server-tx client-tx
</allOneLine> </allOneLine>
A Base64 encoded version of this log entry (without the changes A Base64 encoded version of this log entry (without the changes
required to format it for an internet-draft) is shown below: required to format it for an Internet-Draft) is shown below.
begin-base64 644 clf_record begin-base64 644 clf_record
QTAwMDBGRSwwMDUzMDA1QzAwNUUwMDZEMDA3RDAwOEYwMDlFMDBBMDAwQkEwMEM3MDBF QTAwMDBGRSwwMDUzMDA1QzAwNUUwMDZEMDA3RDAwOEYwMDlFMDBBMDAwQkEwMEM3MDBF
QjAwRjUwMEZFCjAwMDAwMDAwMDAuMDEwICBST1JVVSAgIDEgSU5WSVRFICAgICAgICAt QjAwRjUwMEZFCjAwMDAwMDAwMDAuMDEwICBST1JVVSAgIDEgSU5WSVRFICAgICAgICAt
ICAgICAgIHNpcDoxOTIuMC4yLjEwICAxOTIuMC4yLjEwOjUwNjAgMTkyLjAuMi4yMDA6 ICAgICAgIHNpcDoxOTIuMC4yLjEwICAxOTIuMC4yLjEwOjUwNjAgMTkyLjAuMi4yMDA6
NTY0ODUgICAgICAgc2lwOjE5Mi4wLjIuMTAgIC0gICAgICAgc2lwOjEwMDFAZXhhbXBs NTY0ODUgICAgICAgc2lwOjE5Mi4wLjIuMTAgIC0gICAgICAgc2lwOjEwMDFAZXhhbXBs
ZS5jb206NTA2MCAgICAgICBETDg4MzYwZmE1ZmMgICAgREw3MGRmZjU5MGMxLTEwNzkw ZS5jb206NTA2MCAgICAgICBETDg4MzYwZmE1ZmMgICAgREw3MGRmZjU5MGMxLTEwNzkw
NTE1NTRAZXhhbXBsZS5jb20gICAgIHNlcnZlci10eCAgICAgICBjbGllbnQtdHgK NTE1NTRAZXhhbXBsZS5jb20gICAgIHNlcnZlci10eCAgICAgICBjbGllbnQtdHgK
==== ====
To recover the unencoded file, the Base64 text above may be passed as
input to the following perl script (the output should be redirected
to a file).
#!/usr/bin/perl
use strict;
my $bdata = "";
use MIME::Base64;
while(<>)
{
if (/begin-base64 644 clf_record/ .. /-- ==== --/)
{
if ( m/^\s*[^\s]+\s*$/)
{
$bdata = $bdata . $_;
}
}
}
print decode_base64($bdata);
6. Text Tool Considerations 6. Text Tool Considerations
This format has been designed to allow text tools to easily process This format has been designed to allow text tools to easily process
logs without needing to understand the indexing format. Index lines logs without needing to understand the indexing format. Index lines
may be rapidly discarded by checking the first character of the line: may be rapidly discarded by checking the first character of the line:
index lines will always start with an alphabetical character, while index lines will always start with an alphabetical character, while
field lines will start with a numerical character. field lines will start with a numerical character.
Within a field line, script tools can quickly split fields at the tab Within a field line, script tools can quickly split fields at the tab
characters. The first 12 fields are positional, and the meaning of characters. The first 12 fields are positional, and the meaning of
skipping to change at page 26, line 6 skipping to change at page 26, line 34
[RFC5735] Cotton, M. and L. Vegoda, "Special Use IPv4 Addresses", [RFC5735] Cotton, M. and L. Vegoda, "Special Use IPv4 Addresses",
BCP 153, RFC 5735, January 2010. BCP 153, RFC 5735, January 2010.
[RFC5737] Arkko, J., Cotton, M., and L. Vegoda, "IPv4 Address Blocks [RFC5737] Arkko, J., Cotton, M., and L. Vegoda, "IPv4 Address Blocks
Reserved for Documentation", RFC 5737, January 2010. Reserved for Documentation", RFC 5737, January 2010.
[RFC5952] Kawamura, S. and M. Kawashima, "A Recommendation for IPv6 [RFC5952] Kawamura, S. and M. Kawashima, "A Recommendation for IPv6
Address Text Representation", RFC 5952, August 2010. Address Text Representation", RFC 5952, August 2010.
[W3C.REC-xml-20081126] [W3C.REC-xml-20081126]
Maler, E., Yergeau, F., Bray, T., Sperberg-McQueen, C., Yergeau, F., Sperberg-McQueen, C., Maler, E., Paoli, J.,
and J. Paoli, "Extensible Markup Language (XML) 1.0 (Fifth and T. Bray, "Extensible Markup Language (XML) 1.0 (Fifth
Edition)", World Wide Web Consortium Recommendation REC- Edition)", World Wide Web Consortium Recommendation REC-
xml-20081126, November 2008, xml-20081126, November 2008,
<http://www.w3.org/TR/2008/REC-xml-20081126>. <http://www.w3.org/TR/2008/REC-xml-20081126>.
Authors' Addresses Authors' Addresses
Gonzalo Salgueiro Gonzalo Salgueiro
Cisco Systems Cisco Systems
7200-12 Kit Creek Road 7200-12 Kit Creek Road
Research Triangle Park, NC 27709 Research Triangle Park, NC 27709
 End of changes. 17 change blocks. 
46 lines changed or deleted 65 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/