[Docs] [txt|pdf] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03 04 05 06 07 08 09 RFC 6183

IPFIX Working Group                                         A. Kobayashi
Internet-Draft                                               NTT PF Lab.
Intended status: Informational                                 B. Claise
Expires: January 14, 2010                            Cisco Systems, Inc.
                                                            K. Ishibashi
                                                             NTT PF Lab.
                                                           July 13, 2009


                       IPFIX Mediation: Framework
                draft-ietf-ipfix-mediators-framework-03

Status of this Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.  This document may contain material
   from IETF Documents or IETF Contributions published or made publicly
   available before November 10, 2008.  The person(s) controlling the
   copyright in some of this material may not have granted the IETF
   Trust the right to allow modifications of such material outside the
   IETF Standards Process.  Without obtaining an adequate license from
   the person(s) controlling the copyright in such materials, this
   document may not be modified outside the IETF Standards Process, and
   derivative works of it may not be created outside the IETF Standards
   Process, except to format it for publication as an RFC or to
   translate it into languages other than English.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on January 14, 2010.

Copyright Notice

   Copyright (c) 2009 IETF Trust and the persons identified as the



Kobayashi, et al.       Expires January 14, 2010                [Page 1]

Internet-Draft          IPFIX Mediation Framework              July 2009


   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents in effect on the date of
   publication of this document (http://trustee.ietf.org/license-info).
   Please review these documents carefully, as they describe your rights
   and restrictions with respect to this document.












































Kobayashi, et al.       Expires January 14, 2010                [Page 2]

Internet-Draft          IPFIX Mediation Framework              July 2009


Abstract

   This document describes a framework for IPFIX Mediation.  This
   framework details the IPFIX Mediation reference model and the
   components of an IPFIX Mediator.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Terminology and Definitions  . . . . . . . . . . . . . . . . .  5
   3.  IPFIX/PSAMP Documents Overview . . . . . . . . . . . . . . . .  8
     3.1.  IPFIX Documents Overview . . . . . . . . . . . . . . . . .  8
     3.2.  PSAMP Documents Overview . . . . . . . . . . . . . . . . .  8
   4.  IPFIX Mediation Reference Model  . . . . . . . . . . . . . . .  9
   5.  IPFIX Mediator Components  . . . . . . . . . . . . . . . . . . 14
     5.1.  Collecting Process . . . . . . . . . . . . . . . . . . . . 14
     5.2.  Exporting Process  . . . . . . . . . . . . . . . . . . . . 14
     5.3.  Intermediate Process . . . . . . . . . . . . . . . . . . . 14
       5.3.1.  Intermediate Selection Process . . . . . . . . . . . . 14
       5.3.2.  Intermediate Aggregation Process . . . . . . . . . . . 15
       5.3.3.  Intermediate Anonymization Process . . . . . . . . . . 16
       5.3.4.  Intermediate Correlation Process . . . . . . . . . . . 17
       5.3.5.  Data Record Expiration . . . . . . . . . . . . . . . . 18
     5.4.  Examples . . . . . . . . . . . . . . . . . . . . . . . . . 19
       5.4.1.  Component Combination Examples . . . . . . . . . . . . 19
       5.4.2.  Overview Example . . . . . . . . . . . . . . . . . . . 21
   6.  Specific IPFIX Mediators . . . . . . . . . . . . . . . . . . . 23
   7.  Encoding for IPFIX Message Header  . . . . . . . . . . . . . . 25
   8.  Information Model  . . . . . . . . . . . . . . . . . . . . . . 26
   9.  Security Considerations  . . . . . . . . . . . . . . . . . . . 27
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 28
   11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 29
     11.1. Normative References . . . . . . . . . . . . . . . . . . . 29
     11.2. Informative References . . . . . . . . . . . . . . . . . . 29
   Appendix A.  Acknowledgements  . . . . . . . . . . . . . . . . . . 31
   Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 32














Kobayashi, et al.       Expires January 14, 2010                [Page 3]

Internet-Draft          IPFIX Mediation Framework              July 2009


1.  Introduction

   IPFIX Mediation covers two classes of mediation: content mediation
   for traffic data and transport mediation for transport protocols.
   Content mediation has several manipulations for a sequence of
   records, e.g., aggregation, correlation, filtering, or modification.
   Transport mediation converts other transmitting protocols into IPFIX.
   The motivation for the IPFIX Mediation standard comes from the need
   for a countermeasure to IP traffic growth, a multipurpose traffic
   measurement, and a heterogeneous environment, as described in detail
   in [IPFIX-MD-PS].  This document provides a high-level description of
   an IPFIX Mediator's key components and their functions.

   This document is structured as follows: section 2 describes the
   terminology used in this document, section 3 gives an IPFIX/PSAMP
   document overview, section 4 describes a high-level reference model,
   section 5 describes components and functional features located in an
   IPFIX Mediator, section 6 describes the component models in specific
   IPFIX Mediator types, section 7 describes consideration points of
   encoding for IPFIX Message Headers, and section 8 describes the
   Information Elements used in an IPFIX Mediator.






























Kobayashi, et al.       Expires January 14, 2010                [Page 4]

Internet-Draft          IPFIX Mediation Framework              July 2009


2.  Terminology and Definitions

   The terms in this section are in line with those in the IPFIX
   Protocol specifications [RFC5101] and the PSAMP specification
   document [RFC5476].  The terms Observation Point, Observation Domain,
   Flow Key, Flow Record, Data Record, Exporting Process, Exporter,
   IPFIX Device, Collecting Process, Collector, IPFIX Message, Metering
   Process, Transport Session, Information Element, and Template
   Withdrawal Message are defined in the IPFIX protocol specifications
   [RFC5101].  The terms Packet Report, Sampling, Filtering, PSAMP
   Device, and Configured Selection Fraction are defined in the PSAMP
   specification document [RFC5476].  Furthermore, new terminology to be
   used in the context of IPFIX Mediation is defined in this section.
   All the words in these terms are started with a capital letter in
   this document.

   In this document, we use the generic term "Data Records" for IPFIX
   Flow Records, PSAMP Packet Reports, and Data Records defined by
   Options Templates, unless an explicit distinction is required.

   Transport Session Information

      The Transport Session is specified in [RFC5101].  In SCTP, the
      Transport Session Information is the SCTP association.  In TCP and
      UDP, the Transport Session Information corresponds to a 5-tuple
      {Exporter IP address, Collector IP address, Exporter transport
      port, Collector transport port, and transport protocol}.

   Original Exporter

      An Original Exporter is an IPFIX Device that hosts the Observation
      Points where the metered IP packets are observed.

   IPFIX Mediation

      IPFIX Mediation is the manipulation and conversion of records for
      subsequent export using IPFIX, by applying mediation functions to
      a stream of records.

   The following terms are used in this document to describe the
   architectural entities used by IPFIX Mediation.

   Intermediate Process

      An Intermediate Process takes a sequence of records from a
      Collecting Process, Metering Process, IPFIX File Reader, or
      another Intermediate Process; performs some transformation on
      these records based upon the content of the records themselves,



Kobayashi, et al.       Expires January 14, 2010                [Page 5]

Internet-Draft          IPFIX Mediation Framework              July 2009


      keeps state across multiple records, configuration parameters, or
      other data; and passes a sequence of transformed records on to an
      Exporting Process, IPFIX File Writer, or another Intermediate
      Process.  Typically, an Intermediate Process is hosted by an IPFIX
      Mediator.  Alternatively, an Intermediate Process may be hosted by
      an Original Exporter.

   This document describes specific Intermediate Processes below.
   However, this is not an exhaustive list.

   Intermediate Aggregation Process

      An Intermediate Aggregation Process is an Intermediate Process
      that aggregates records based upon a set of Flow Keys or functions
      applied to fields from the record (e.g., binning, subnet
      aggregation).

   Intermediate Correlation Process

      An Intermediate Correlation Process is an Intermediate Process
      which adds information to records, noting correlations among them,
      or generates new records with correlated data from multiple
      records (e.g., the production of bidirectional flow records from
      unidirectional flow records).

   Intermediate Selection Process

      An Intermediate Selection Process is an Intermediate Process that
      selects records from a sequence based upon criteria evaluated
      record values and passes only those records that match the
      criteria (e.g., filtering only records from a given network to a
      given Collector).

   Intermediate Anonymization Process

      An Intermediate Anonymization Process is an Intermediate Process
      that transforms records in order to anonymize them, to protect the
      identity of the entities described by the records (e.g., by
      applying prefix-preserving pseudonymization of IP addresses).

   IPFIX Mediator

      An IPFIX Mediator is an IPFIX Device that provides mediation
      capabilities by receiving records from some data source, hosting
      zero or more Intermediate Processes to transform those records,
      and exporting those records in IPFIX Messages via an Exporting
      Process.  In the common case, an IPFIX Mediator receives records
      from a Collecting Process but could also receive records from data



Kobayashi, et al.       Expires January 14, 2010                [Page 6]

Internet-Draft          IPFIX Mediation Framework              July 2009


      sources not encoded using IPFIX, e.g., in the case of NetFlow V9
      protocol translation.

   Specific types of IPFIX Mediators are defined below.

   IPFIX Proxy

      An IPFIX Proxy is an IPFIX Mediator that relays incoming IPFIX
      Messages or messages in other protocols to one or more Collectors.
      It can provide transport protocol mediation and re-encoding.

   IPFIX Concentrator

      An IPFIX Concentrator is an IPFIX Mediator that receives data from
      one or more Exporters and sends them to a single Collector,
      optionally transforming the records using zero or more
      Intermediate Processes on the way.

   IPFIX Distributor

      An IPFIX Distributor is an IPFIX Mediator that receives data from
      one or more Exporters and sends them to one or more Collectors,
      deciding which Collector(s) to send each record to based upon the
      decision of an Intermediate Process.

   IPFIX Masquerading Proxy

      An IPFIX Masquerading Proxy is an IPFIX Mediator that receives
      data from one or more Exporters and sends them to a single
      Collector, using one or more Intermediate Processes to screen out
      parts of records according to configured policies, in order to
      protect the privacy of the network's end users or sensitive data
      of the exporting organization.


















Kobayashi, et al.       Expires January 14, 2010                [Page 7]

Internet-Draft          IPFIX Mediation Framework              July 2009


3.  IPFIX/PSAMP Documents Overview

3.1.  IPFIX Documents Overview

   The IPFIX protocol [RFC5101] provides network administrators with
   access to IP flow information.  The architecture for the export of
   measured IP flow information from an IPFIX Exporting Process to a
   Collecting Process is defined in [RFC5470], per the requirements
   defined in [RFC3917].  The IPFIX protocol [RFC5101] specifies how
   IPFIX Data Records and Templates are carried via a number of
   transport protocols from IPFIX Exporting Processes to IPFIX
   Collecting Processes.  IPFIX has a formal description of IPFIX
   Information Elements, their names, types, and additional semantic
   information, as specified in [RFC5102].  [IPFIX-MIB] specifies the
   IPFIX Management Information Base.  Finally, [RFC5472] describes what
   types of applications can use the IPFIX protocol and how they can use
   the information provided.  It furthermore shows how the IPFIX
   framework relates to other architectures and frameworks.  The storage
   of IPFIX Messages in a file is specified in [IPFIX-FILE].

3.2.  PSAMP Documents Overview

   The framework for packet selection and reporting [RFC5474] enables
   network elements to select subsets of packets by statistical and
   other methods and to export a stream of reports on the selected
   packets to a Collector.  The set of packet selection techniques
   (sampling, filtering, and hashing) standardized by PSAMP is described
   in [RFC5475].  The PSAMP protocol [RFC5476] specifies the export of
   packet information from a PSAMP Exporting Process to a Collector.
   Like IPFIX, PSAMP has a formal description of its Information
   Elements, their names, types, and additional semantic information.
   The PSAMP information model is defined in [RFC5477].  [PSAMP-MIB]
   describes the PSAMP Management Information Base.


















Kobayashi, et al.       Expires January 14, 2010                [Page 8]

Internet-Draft          IPFIX Mediation Framework              July 2009


4.  IPFIX Mediation Reference Model

   The figure below shows the high-level IPFIX Mediation component model
   based on [RFC5470].  This figure covers the various possible
   scenarios that can exist in an IPFIX measurement system.

   +---------------------------+    +---------------------------+
   | Collector 1               |    | Collector N               |
   |[Collecting Process(es)]   |....|[Collecting Process(es)]   |
   +---------------------------+    +---------------------------+
                    ^    ^              ^  ^
                    |    |              |  |
                    |    +------....----+  |
                    |    |                 |
                      IPFIX (Data Records)
                    |    |                 |
   +----------------+----+-----+    +-------+-------------------+
   |IPFIX Mediator 1           |    |IPFIX Mediator N           |
   |[Exporting Process(es)]    |    |[Exporting Process(es)]    |
   |[Intermediate Process(es)] |....|[Intermediate Process(es)] |
   |[Collecting Process(es)]   |    |[Collecting Process(es)]   |
   +---------------------------+    +---------------------------+
                    ^    ^               ^
                    |    |               |
                    |    +------....-----+
                    |                    |
                     IPFIX (Data Records)
                    |                    |
   +----------------+----------+    +----+----------------------+
   |IPFIX Original Exporter 1  |    |IPFIX Original Exporter N  |
   |[Exporting Process(es)]    |    |[Exporting Process(es)]    |
   |[Metering Process(es)]     |....|[Metering Process(es)]     |
   |[Observation Point(s)]     |    |[Observation Point(s)]     |
   +---------------------------+    +---------------------------+
               ^ ^                        ^ ^
               | |                        | |
            Packets coming to Observation Points

   Figure A: IPFIX Mediation Component Model Overview.

   The functional components within each device are indicated within
   brackets [].









Kobayashi, et al.       Expires January 14, 2010                [Page 9]

Internet-Draft          IPFIX Mediation Framework              July 2009


   The figure below shows the basic IPFIX Mediator component model.  An
   IPFIX Mediator is defined as consisting of one or more Collecting
   Processes, zero or more Intermediate Processes, and one or more
   Exporting Processes.  Basically, an IPFIX Mediator, i.e., IPFIX
   Proxy, IPFIX Masquerading Proxy, IPFIX Distributor, and IPFIX
   Concentrator, are composed of these components.

                  IPFIX (Data Records)
                              ^
                            ^ |
   +------------------------|-|---------------------+
   | IPFIX Mediator         | |                     |
   |                        | |                     |
   |  .---------------------|-+-------------------. |
   | .----------------------+--------------------.| |
   | |          Exporting Process(es)            |' |
   | '----------------------^--------------------'  |
   |                        | |                     |
   |  .---------------------|-+-------------------. |
   | .----------------------+--------------------.| |
   | |          Intermediate Process(es)         |' |
   | '----------------------^--------------------'  |
   |                        | |                     |
   |  .---------------------|-+-------------------. |
   | .----------------------+--------------------.| |
   | |          Collecting Process(es)           |' |
   | '----------------------^--------------------'  |
   +------------------------|-|---------------------+
                            |
                  IPFIX (Data Records)

   Figure B: Basic IPFIX Mediator Component Model.



















Kobayashi, et al.       Expires January 14, 2010               [Page 10]

Internet-Draft          IPFIX Mediation Framework              July 2009


   In another case, an IPFIX Mediator, i.e., IPFIX Proxy, receives
   traffic records from other transmitting protocols, e.g., NetFlow.
   This document does not make any particular assumption on how traffic
   records are transferred to an IPFIX Mediator regardless of whether
   the traffic record is flow-based or packet-based.  The figure below
   shows the IPFIX Mediator component model in the case of IPFIX
   protocol conversion.

                  IPFIX (Data Records)
                              ^
                            ^ |
   +------------------------|-|---------------------+
   | IPFIX Mediator         | |                     |
   |  .---------------------|-+-------------------. |
   | .----------------------+--------------------.| |
   | |          Exporting Process(es)            |' |
   | '----------------------^--------------------'  |
   +------------------------|-----------------------+
                            | Traffic record
   +------------------------|-----------------------+
   |          +-------------+----------+            |
   |.---------+-----------.  .---------+-----------.|
   || Observation Point 1 |..| Observation Point N ||
   |'---------^-----------'  '---------^-----------'|
   +----------|------------------------|------------+
              |                        |
            Packets coming to Observation Points

   Figure C: IPFIX Mediator Component Model in IPFIX Protocol
   Conversion.





















Kobayashi, et al.       Expires January 14, 2010               [Page 11]

Internet-Draft          IPFIX Mediation Framework              July 2009


   An Intermediate Process hosted in an Original Exporter receives Data
   Records from IPFIX Metering Processes or PSAMP Metering Processes.
   An Original Exporter with IPFIX Mediation is modeled as follows.

                     IPFIX (Data Records)
                               ^ ^
   +---------------------------|-|------------------------+
   | Original Exporter         | |                        |
   |                           | |                        |
   |     .---------------------|-+-------------------.    |
   |    .----------------------+--------------------.|    |
   |    |           Exporting Process(es)           |'    |
   |    '----------------------^--------------------'     |
   |                           | |                        |
   |     .---------------------|-+-------------------.    |
   |    .----------------------+--------------------.|    |
   |    |          Intermediate Process(es)         |'    |
   |    '---------^-----------------------^---------'     |
   |              |      Data Records     |               |
   |   .----------+---------.   .---------+----------.    |
   |   | Metering Process 1 |...| Metering Process N |    |
   |   '----------^---------'   '---------^----------'    |
   |              |                       |               |
   |  .------------+--------.   .---------+-----------.   |
   |  | Observation Point 1 |...| Observation Point N |   |
   |  '------------^--------'   '---------^-----------'   |
   +--------------|-----------------------|---------------+
                  |                       |
            Packets coming to Observation Points

   Figure D: IPFIX Mediation Component Model at Original Exporter.




















Kobayashi, et al.       Expires January 14, 2010               [Page 12]

Internet-Draft          IPFIX Mediation Framework              July 2009


   An Intermediate Process may be hosted with an IPFIX File Reader
   and/or Writer.  The following figure shows an IPFIX Mediation
   component model with an IPFIX File Writer and/or Reader.

                   IPFIX (Data Records)
                               ^
                             ^ |
      .----------------------|-+--------------------.
     .-----------------------+---------------------.|
     |  Exporting Process(es) / IPFIX File Writer  |'
     '-----------------------^---------------------'
                             | |
      .----------------------|-+--------------------.
     .-----------------------+---------------------.|
     |          Intermediate Process(es)           |'
     '-----------------------^---------------------'
                             | |
      .----------------------|-+--------------------.
     .-----------------------+---------------------.|
     | Collecting Process(es) / IPFIX File Reader  |'
     '-----------------------^---------------------'
                             |
                   IPFIX (Data Records)

   Figure E: IPFIX Mediation Component Model with IPFIX File Writer/
   Reader.

























Kobayashi, et al.       Expires January 14, 2010               [Page 13]

Internet-Draft          IPFIX Mediation Framework              July 2009


5.  IPFIX Mediator Components

   This section describes IPFIX Mediator Components along with examples.

5.1.  Collecting Process

   A Collecting Process, described in [RFC5101], receives Data Records
   with information relating to their treatment in the Metering Process
   and the Exporting Process in an Original Exporter, e.g., sampling
   parameters, IPFIX Message header information, and Transport Session
   Information.  The Collecting Process transmits the set of data to one
   or more components: Intermediate Processes, Exporting Process, and
   some applications.  In other words, a Collecting Process may
   duplicate received Data Records and transmit them to one or more
   components in sequence or in parallel.

5.2.  Exporting Process

   An Exporting Process, described in [RFC5101], sends Data Records in
   the form of IPFIX Messages to one or more Collectors.  The Exporting
   Process also needs to send subsidiary information (e.g., sampling
   parameters and a set of Flow Keys) in the form of a Data Record when
   receiving records from other transmitting protocols as well.

5.3.  Intermediate Process

   An Intermediate Process is a key process for content mediation and
   generates new sets of Data Records or entire IPFIX Messages from
   input records with context information (e.g., "Export Time" and
   "Observation Domain ID").  In the case of a combination of
   Intermediate Processes, the output data from one Intermediate Process
   forms the input data for the succeeding Intermediate Process.  The
   following subsections show the different specific Intermediate
   Process details.

5.3.1.  Intermediate Selection Process

   An Intermediate Selection Process selects records; this is analogous
   to the PSAMP Selection Process described in [RFC5475].  The
   difference is that the Intermediate Selection Process takes received
   records, regardless of whether they are Flow Records or Packet
   Reports, rather than observed packets.  In the case of filtering, an
   Intermediate Selection Process determines which input records are
   selected by matching them under a filtering policy and then transmits
   them to other components.






Kobayashi, et al.       Expires January 14, 2010               [Page 14]

Internet-Draft          IPFIX Mediation Framework              July 2009


5.3.2.  Intermediate Aggregation Process

   An Intermediate Aggregation Process creates aggregated Flow Records
   from input records and then transmits them to other components.
   There are three types of aggregation into this process.

   Flow Key Field Selection

      The Intermediate Aggregation Process gathers records within a
      given interval time and then merges records that have the same
      values of given Flow Key fields.  Decreasing the number of Flow
      Key fields in aggregation results in more aggregated Flow Records
      being created.  If the input Flow Records include the
      "flowKeyIndicator" field described in [RFC5102], the Intermediate
      Aggregation Process needs to modify its value.

      In addition, the Intermediate Aggregation Process can create
      statistical data and subsidiary information related to a set of
      aggregated Flow Records.  Examples include the given interval
      time, a new set of Flow Keys, and the number of input Data Records
      belonging to an aggregated Flow Record.

   Time Composition

      Time composition is defined as aggregation of Data Records within
      a given interval time without changing their Flow Key(s).  The
      Intermediate Aggregation Process may also compute Data Records
      statistics, such as maximum and minimum values of per-flow
      counters.  The Time Composition provides some advantages.

      *  reducing the number of Flow Records for long-running Flows

      *  computing the active time period for long-running Flows

      *  revealing the time series behavior of traffic volume within an
         active time

         Short period Flow Records created at a Metering Process by
         configuring a short active time, e.g., 1 or 10 sec, are merged
         at an Intermediate Aggregation Process within a certain time
         period, e.g., 60 or 300 sec.  While merging, the Intermediate
         Aggregation Process computes new metrics such as the maximum
         and minimum.  It produces more precise maximum and minimum
         values without increasing the number of Flow Records on a
         Collector.  When some traffic requires timely traffic
         monitoring and other traffic does not, a combination of the
         Intermediate Selection Process and Intermediate Aggregation
         Process is useful, as described in section 5.4.



Kobayashi, et al.       Expires January 14, 2010               [Page 15]

Internet-Draft          IPFIX Mediation Framework              July 2009


   Space Composition

      Space composition is defined as aggregation on a larger
      Observation Domain or across a set of Observation Points or
      Observation Domains.  Generally, Flow Key fields are included in a
      Data Record.  In that case, other properties that are not included
      in a Data Record, such as the Exporter IP address or Observation
      Domain ID, may be Flow Key fields.

      As another approach, an identifier indicating a spatial
      Observation Domain can also become a new Flow Key. For example, an
      identifier indicates an area on an ISP network or a link
      aggregation interface composed of some physical interfaces.  The
      identifier also makes a relation to a set of values of a specified
      field in the input Data Records by the configuring rule.  After
      converting the values of the specified field to the identifier,
      the Intermediate Aggregation Process can create aggregated Flow
      Records by a general aggregation process by using the identifier
      as a Flow Key field.

5.3.3.  Intermediate Anonymization Process

   An Intermediate Anonymization Process modifies the value of specified
   fields or screens out specified fields without changing their Flow
   Key(s).  The Intermediate Anonymization Process also needs to modify
   the value of the "flowKeyIndicator" when modifying the data structure
   of an incoming Template.

   Deleting specified fields

      The Intermediate Anonymization Process deletes existing fields in
      accordance with instruction rules, which indicate whether a
      specified Information Element should be deleted.

      Applicable examples include hiding network topology information
      and private information.  In the case of feeding Data Records to
      end customers, the Intermediate Anonymization Process avoids
      disclosing vulnerabilities by deleting fields, e.g.,
      "ipNextHopIP{v4|v6}Address", "bgpNextHopIP{v4|v6}Address",
      "bgp{Next|Prev}AdjacentAsNumber", and "mplsLabelStackSection",
      described in [RFC5102].

   Anonymizing value of specified fields

      The Intermediate Anonymization Process modifies the value of
      specified fields.





Kobayashi, et al.       Expires January 14, 2010               [Page 16]

Internet-Draft          IPFIX Mediation Framework              July 2009


      Applicable examples include anonymizing customers' private
      information, such as IP address and port number, in accordance
      with a privacy protection policy.  The Intermediate Anonymization
      Process may also report anonymized fields and the anonymization
      method as subsidiary information.

5.3.4.  Intermediate Correlation Process

   An Intermediate Correlation Process creates new metrics, counters,
   attributes, or packet property parameters by evaluating the
   correlation among sets of Data Records or among Data Records and
   other meta data after gathering sets of Data Records during a given
   interval time.  After producing new values, the Intermediate
   Correlation Process adds new fields to the Data Records or creates a
   new Data Record.  The Intermediate Correlation Process also needs to
   modify the value of the "flowKeyIndicator" when modifying the data
   structure of an incoming Template.  The Intermediate Correlation
   Process indicates a special case of the Intermediate Aggregation
   Process.  Typical examples are as follows:

   o  One-to-one correlation between Data Records

      *  One way delay, Packet delay variation in [RFC5481]
         The Intermediate Correlation Process gathers a pair of Packet
         Reports exported from different Exporters indicating the same
         packet.  The metrics follow from the correlation of the
         timestamp value on both of the Packet Reports.

      *  Packet inter-arrival time or jitter
         The Intermediate Correlation Process gathers consecutive Packet
         Reports exported from an Exporter.

      *  Rate-limiting ratio, compression ratio, optimization ratio,
         etc.
         The data values follow from the correlation of Data Records
         within a single Flow observed on the incoming/outgoing points
         of a WAN interface.

   o  Correlation amongst Data Records

      *  Average/maximum/minimum packets, bytes, one way delay, packet
         loss, etc.
         The data values follow from the correlation of multiple Data
         Records while the Intermediate Aggregation Process executes.

   o  Correlation between Data Record and other meta data

      Typical examples are derived packet property parameters described



Kobayashi, et al.       Expires January 14, 2010               [Page 17]

Internet-Draft          IPFIX Mediation Framework              July 2009


      in [RFC5102].  The parameters are retrieved from a database etc.
      based on the value of the specified field in an input Data Record.
      Doing that can compensate for traditional exporting devices or
      probes that are unable to add packet property parameters.
      Therefore, Collectors do not need to recognize the difference
      among implementations of routers from several vendors or among
      Exporter types, such as router, switch, or probe.  Typical derived
      packet property parameters are as follows:

      *  "bgpNextHop{IPv4|IPv6}Address" described in [RFC5102]
         The address indicates the egress router of a network domain.
         That is useful for making a traffic matrix that covers the
         whole network domain.

      *  BGP Communities attribute
         The attribute indicates tagging for routes of geographical and
         topological information and source type (e.g., transit, peer,
         or customer) as described in [RFC4384].  Therefore, network
         administrators can monitor the geographically-based or source
         type-based traffic volume by correlating the attribute.

      *  "mplsVpnRouteDistinguisher" described in [RFC5102]
         The value indicates the VPN customer's identification, which
         cannot be extracted from the core router in MPLS networks.
         Therefore, network administrators can monitor the customer-
         based traffic volume on even core routers.

5.3.5.  Data Record Expiration

   An Intermediate Aggregation Process and an Intermediate Correlation
   Process need to have expiration conditions to export cached Data
   Records.  In the case of the Metering Process in an Original
   Exporter, these conditions are described in [RFC5470].  In the case
   of the Intermediate Process, these conditions are as follows.

   o  If there are no input Data Records belonging to a cached Flow for
      a certain time period, aggregated Flow Records will expire.  This
      time period should be configurable at the Intermediate Process.

   o  If the Intermediate Process experiences resource constraints,
      aggregated Flow Records may prematurely expire (e.g., lack of
      memory to store Flow Records).

   o  For long-running Flows, the Intermediate Process should cause the
      Flow to expire on a regular basis or on the basis of an expiration
      policy.  This periodicity or expiration policy should be
      configurable at the Intermediate Process.




Kobayashi, et al.       Expires January 14, 2010               [Page 18]

Internet-Draft          IPFIX Mediation Framework              July 2009


   The Intermediate Correlation Process has special cases.  Cached Data
   Records may be discarded when they prematurely expire and the
   Intermediate Correlation Process cannot compute their correlation.
   For example, an Intermediate Correlation Process computing one way
   delay may discard the cached Packet Report due to computation failure
   when receiving one Packet Report and not receiving another Packet
   Report until expiration of the cached Data Record.

5.4.  Examples

5.4.1.  Component Combination Examples

   The combination of some components (Intermediate Process, Exporting
   Process, Collecting Process, IPFIX File Writer and Reader) provides
   useful applications.  This subsection describes examples as follows.

   Data-based Collector Selection

      The combination of one or more Intermediate Selection Processes
      and Exporting Processes can determine to which Collector input
      Data Records are exported.  Applicable examples include exporting
      Data Records to a dedicated Collector on the basis of customer or
      organization peering.  For example, an Intermediate Selection
      Process selects Data Records on the basis of the peering
      autonomous system number, and an Exporting Process sends them to a
      dedicated Collector, as shown in the following figure.

             .----------------------.   .------------.
             | Intermediate         |   | Exporting  |
             |  Selection Process 1 |   |  Process 1 |
          +--+--- Peering AS #10 ---+-->|            +--> Collector 1
          |  '----------------------'   '------------'
          |  .----------------------.   .------------.
   Data   |  | Intermediate         |   | Exporting  |
   Record |  |  Selection Process 2 |   |  Process 2 |
   -------+--+--- Peering AS #20 ---+-->|            +--> Collector 2
          |  '----------------------'   '------------'
          |  .----------------------.   .------------.
          |  | Intermediate         |   | Exporting  |
          |  |  Selection Process 3 |   |  Process 3 |
          +--+--- Peering AS #30 ---+-->|            +--> Collector 3
             '----------------------'   '------------'

      Figure F: Data-based Collector Selection Example.







Kobayashi, et al.       Expires January 14, 2010               [Page 19]

Internet-Draft          IPFIX Mediation Framework              July 2009


   Flow Selection and Aggregation

      The combination of one or more Intermediate Selection Processes
      and Intermediate Aggregation Processes can efficiently reduce the
      amount of Flow Records.  For example, an Intermediate Selection
      Process selects small Flows consisting of a small number of
      packets and then transmits them to an Intermediate Aggregation
      Process.  Another Intermediate Selection Process selects other
      Flow Records and then transmits them to an Exporting Process, as
      shown in the following figure.  This results in aggregation on the
      basis of the distribution of the number of packets per Flow.

            .------------------.  .--------------.  .------------.
            | Intermediate     |  | Intermediate |  | Exporting  |
            |   Selection      |  |  Aggregation |  |    Process |
            |        Process 1 |  |     Process  |  |            |
          +-+ packetDeltaCount +->|              +->|            |
          | |             <= 5 |  |              |  |            |
   Data   | '------------------'  '--------------'  |            |
   Record | .------------------.                    |            |
   -------+ | Intermediate     |                    |            |
          | |   Selection      |                    |            |
          | |        Process 2 |                    |            |
          +-+ packetDeltaCount +------------------->|            |
            |              > 5 |                    |            |
            '------------------'                    '------------'

      Figure G: Flow Selection and Aggregation Example.

   IPFIX File Writer/Reader

      The IPFIX File Writer/Reader on an IPFIX Mediator also complies
      with [IPFIX-FILE].  The IPFIX File Writer stores input Data
      Records from any process in a file system.  When input Data
      Records include irrelevant Information Elements, an Intermediate
      Anonymization Process can delete these fields before the IPFIX
      File Writer handles them, as shown in the following figure.

         .---------------.  .---------------.  .-------------.
         | Collecting    |  | Intermediate  |  | IPFIX       |
   IPFIX |      Process  |  | Anonymization |  |   File      |
   ----->|               +->|       Process +->|      Writer |
         '---------------'  '---------------'  '-------------'

      Figure H: IPFIX Mediation Example with IPFIX File Writer.  In
      contrast, the IPFIX File Reader retrieves stored Data Records when
      administrators want to retrieve past Data Records from a given
      time period.  If the data structure of output Data Records from



Kobayashi, et al.       Expires January 14, 2010               [Page 20]

Internet-Draft          IPFIX Mediation Framework              July 2009


      the IPFIX File Reader is different from what administrators want,
      an Intermediate Anonymization Process and Intermediate Correlation
      Process can modify the data structure, as shown in the following
      figure.

   .-------------.  .---------------.  .---------------.  .-----------.
   | IPFIX       |  | Intermediate  |  | Intermediate  |  | Exporting |
   |   File      |  | Anonymization |  |   Correlation |  |   Process |
   |      Reader +->|       Process +->|       Process +->|           |
   '-------------'  '---------------'  '---------------'  '-----------'

      Figure I: IPFIX Mediation Example with IPFIX File Reader.

5.4.2.  Overview Example

   As an example in the case of the IPFIX Mediator having different
   Intermediate Process types, a Collecting Process/IPFIX File Reader
   replicates Data Records, if necessary, and transmits them to a
   suitable Intermediate Process/Exporting Process.  An example figure
   is shown below.































Kobayashi, et al.       Expires January 14, 2010               [Page 21]

Internet-Draft          IPFIX Mediation Framework              July 2009


                        IPFIX           IPFIX               IPFIX
                          ^               ^                   ^
                          |               |                   |
    .------------.  .-----+-------. .-----+-------.    .------+------.
    | IPFIX File |  | Exporting   | | Exporting   |    | Exporting   |
    |  Writer    |  |  Process 1  | |  Process 2  |....|  Process N  |
    '-----^-^----'  '-----^-------' '-----^-------'    '------^------'
          | |             |               |                   |
          | +-------------+               |                   |
          :          Flow Records / Packet Reports            :
   .------+-------. .-----+--------. .----+---------.         |
   | Intermediate | | Intermediate | | Intermediate |         |
   | Anonymization| | Correlation  | | Aggregation  |         |
   | Process N    | | Process N    | |  Process N   |         |
   '------|-------' '------|-------' '-----|-|------'         |
          |                +---------------+ |                |
          :                :                 :                :
   .------+-------. .------+-------. .-------+------.         |
   | Intermediate | | Intermediate | | Intermediate |         |
   | Selection    | | Selection    | | Selection    |         |
   | Process 1    | | Process 2    | |  Process 3   |         |
   '------|-|-----' '------|-------' '-----|--------'         |
          | +--------------+               | +----------------+
          |                |               | |                |
          :          Flow Records / Packet Reports            :
   .------+------. .-------+-----.   .-----+-+-----.    .-----+------.
   | Collecting  | | Collecting  |   | Collecting  |    | IPFIX File |
   |  Process 1  | |  Process 2  |...|  Process N  |    |  Reader    |
   '------^------' '------^------'   '------^------'    '------------'
          |               |                 |
        IPFIX           IPFIX             IPFIX

   Figure J: IPFIX Mediation Functional Block Examples.


















Kobayashi, et al.       Expires January 14, 2010               [Page 22]

Internet-Draft          IPFIX Mediation Framework              July 2009


6.  Specific IPFIX Mediators

   This document intends to avoid constraining the component models of
   IPFIX Mediators.  However, typical component models of specific IPFIX
   Mediators can be expected.  This section describes the component
   models of the specific types to elaborate the framework of IPFIX
   Mediation.

   The figure below shows the component models in each specific type.
   Specific IPFIX Mediators are composed of the following components:
   Collecting Process (C), Intermediate Process (I), and Exporting
   Process (E).  The components within brackets [] indicate that there
   are zero or more hosted.  The components without brackets indicate
   one or more.

                    +------>       --->-----+       +------>
         +----------+--+                 +--+-------+--+
   --->--+-[C]------E--+--->       --->--+--C---I---E--+--->
         +----------+--+                 +--+-------+--+
                    +------>       --->-----+       +------>

           IPFIX Proxy                 IPFIX Distributor


   --->-----+                      --->-----+
         +--+----------+                 +--+----------+
   --->--+--C--[I]--E--+--->       --->--+--C---I---E--+--->
         +--+----------+                 +--+----------+
   --->-----+                      --->-----+

       IPFIX Concentrator          IPFIX Masquerading Proxy

   Figure K: Component Models in Specific IPFIX Mediators.


















Kobayashi, et al.       Expires January 14, 2010               [Page 23]

Internet-Draft          IPFIX Mediation Framework              July 2009


   Below is a summary table for specific IPFIX Mediator types.  The
   abbreviation "IP" stands for Intermediate Process.

   +--------------+------------+---------------+---------------------+
    Mediator Type  Number of    IP Type          Number of IPFIX
                   hosted IPs                    Transport Sessions
   +==============+============+===============+=====================+
    Proxy          zero                         incoming: zero(*)
                                                               or one
                                                outgoing: one or more
   +--------------+------------+---------------+---------------------+
    Distributor    one or more  Selection       incoming: one or more
                                                outgoing: one or more
   +--------------+------------+---------------+---------------------+
    Concentrator   zero or more Aggregation     incoming: one or more
                                Correlation     outgoing: one
   +--------------+------------+---------------+---------------------+
    Masquerading   one or more  Anonymization   incoming: one or more
    Proxy                                       outgoing: one
   +--------------+------------+---------------+---------------------+
   (*) When an IPFIX Proxy converts IPFIX from other protocols.

   Figure L: IPFIX Mediator Type Summary Table.

   An IPFIX Proxy or an IPFIX Concentrator manipulates incoming IPFIX
   Messages without hosting an Intermediate Process when executing
   transport mediation (e.g., converting NetFlow into IPFIX).  In that
   case, IPFIX Mediators manage the mapping information about Transport
   Sessions, Observation Domain IDs, and Template IDs on the incoming/
   outgoing sides.





















Kobayashi, et al.       Expires January 14, 2010               [Page 24]

Internet-Draft          IPFIX Mediation Framework              July 2009


7.  Encoding for IPFIX Message Header

   This section describes consideration points of encoding for the IPFIX
   Message Header.  The IPFIX Message Header described in [RFC5101]
   includes Export Time, Sequence Number, and Observation Domain ID
   fields.  Following are some consideration points:

   Export Time

      An IPFIX Mediator sets the Export Time in two ways:

      *  Case 1: keeping the field value of incoming Transport Sessions

      *  Case 2: setting the time at which an IPFIX Message leaves the
         IPFIX Mediator

      In case 2, the IPFIX Mediator needs to handle any delta time stamp
      fields, such as "flowStartDeltaMicroseconds" and
      "flowEndDeltaMicroseconds", described in [RFC5102].

   Sequence Number

      In the case of an IPFIX Proxy relaying a one-to-one Transport
      Session, the IPFIX Proxy needs to handle the Sequence Number value
      when the incoming Transport Session shuts down and starts.

   Observation Domain ID

      An IPFIX Mediator can set the Observation Domain ID independently
      of the incoming Observation Domain ID.  There are two
      consideration points:

      *  Case 1: relaying an IPFIX Message after replacing each incoming
         Observation Domain ID with a new value in case of an IPFIX
         Proxy and an IPFIX Concentrator

      *  Case 2: aggregating incoming Flow Records in case of an IPFIX
         Concentrator

      In case 1, an IPFIX Mediator needs to set the appropriate scope
      fields in Data Records defined in Options Template Records when
      the incoming Observation Domain IDs used as the scope fields.  In
      case 2, according to the description of [RFC5101], an IPFIX
      Concentrator needs to set a value of 0 for the Observation Domain
      ID.  In that case, the IPFIX Concentrator can add new field to
      Flow Record instead of the Observation Domain ID.  The field
      indicates the largest set of Observation Points for aggregated
      Flow Record.



Kobayashi, et al.       Expires January 14, 2010               [Page 25]

Internet-Draft          IPFIX Mediation Framework              July 2009


8.  Information Model

   IPFIX Mediation reuses the general information models from [RFC5102]
   and [RFC5477].  However, several Intermediate Processes would require
   additional Information Elements as follows:

   o  Number of input Data Records belonging to output aggregated Flow
      Records

   o  New observation domain information instead of Observation Domain
      ID in IPFIX Concentrator

   o  Maximum and minimum values for packet count and octet count

   o  Some metrics related to network performance, e.g., one way delay,
      and packet inter-arrival time, etc.

   o  Anonymization method and report on the anonymized fields

   o  Report on the applied treatment items in IPFIX Mediation































Kobayashi, et al.       Expires January 14, 2010               [Page 26]

Internet-Draft          IPFIX Mediation Framework              July 2009


9.  Security Considerations

   An IPFIX measurement system must also prevent the security threats
   related to IPFIX Mediation that follow as well as the security
   threats described in the security consideration section in [RFC5101].

   o  Attacks against IPFIX Mediators

      IPFIX Mediators need to prevent unauthorized access or denial-of-
      service (DoS) attacks from untrusted public networks.  One
      solution is for IPFIX Mediators to host the packet filter function
      to reject malicious packets at an outside interface.

   o  Man-in-the-middle attacks by untrusted IPFIX Mediators

      The Collector-Mediator-Exporter structure model would increase the
      risk of man-in-the-middle attacks.  One solution is that IPFIX
      Collectors and Exporters must verify trusted IPFIX Mediators to
      prevent connection to untrusted IPFIX Mediators.

   o  Configuration of IPFIX Mediation

      In the case of IPFIX Distributors and IPFIX Masquerading Proxies,
      an accidental misconfiguration and unauthorized access to
      configuration data could lead to the crucial problem of disclosure
      of confidential traffic data.
      To eliminate these risks, IPFIX Mediators must provide the
      authentication function for authorized administrators and the
      facilities to help in tracing configuration changes to their
      origins.





















Kobayashi, et al.       Expires January 14, 2010               [Page 27]

Internet-Draft          IPFIX Mediation Framework              July 2009


10.  IANA Considerations

   This document has no actions for IANA.
















































Kobayashi, et al.       Expires January 14, 2010               [Page 28]

Internet-Draft          IPFIX Mediation Framework              July 2009


11.  References

11.1.  Normative References

   [RFC5101]  Claise, B., "Specification of the IP Flow Information
              Export (IPFIX) Protocol for the Exchange of IP Traffic
              Flow Information", January 2008.

   [RFC5476]  Claise, B., Quittek, J., and A. Johnson, "Packet Sampling
              (PSAMP) Protocol Specifications", March 2009.

11.2.  Informative References

   [IPFIX-FILE]
              Trammell, B., Boschi, E., Mark, L., Zseby, T., and A.
              Wagner, "An IPFIX-Based File Format",
              draft-ietf-ipfix-file-04 (work in progress) , July 2009.

   [IPFIX-MD-PS]
              Kobayashi, A., Claise, B., Nishida, H., Sommer, C.,
              Dressler, F., and E. Stephan, "IPFIX Mediation: Problem
              Statement",
              draft-ietf-ipfix-mediation-problem-statement-03 (work in
              progress) , April 2009.

   [IPFIX-MIB]
              Dietz, T., Claise, B., and A. Kobayashi, "Definitions of
              Managed Objects for IP Flow Information Export",
              draft-ietf-ipfix-mib-06 (work in progress) , March 2009.

   [PSAMP-MIB]
              Dietz, T. and B. Claise, "Definitions of Managed Objects
              for Packet Sampling", draft-ietf-psamp-mib-06 (work in
              progress) , June 2006.

   [RFC3917]  Quittek, J., Zseby, T., Claise, B., and S. Zander,
              "Requirements for IP Flow Information Export(IPFIX)",
              October 2004.

   [RFC4384]  Meyer, D., "BGP Communities for Data Collection",
              February 2006.

   [RFC5102]  Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
              Meyer, "Information Model for IP Flow Information Export",
              January 2008.

   [RFC5470]  Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
              "Architecture for IP Flow Information Export", March 2009.



Kobayashi, et al.       Expires January 14, 2010               [Page 29]

Internet-Draft          IPFIX Mediation Framework              July 2009


   [RFC5472]  Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IPFIX
              Applicability", March 2009.

   [RFC5474]  Duffield, N., "A Framework for Packet Selection and
              Reporting", March 2009.

   [RFC5475]  Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
              Raspall, "Sampling and Filtering Techniques for IP Packet
              Selection", March 2009.

   [RFC5477]  Dietz, T., Claise, B., Aitken, P., Dressler, F., and G.
              Carle, "Information Model for Packet Sampling Exports",
              March 2009.

   [RFC5481]  Morton, A. and B. Claise, "Packet Delay Variation
              Applicability Statement", March 2009.



































Kobayashi, et al.       Expires January 14, 2010               [Page 30]

Internet-Draft          IPFIX Mediation Framework              July 2009


Appendix A.  Acknowledgements

   We would like to thank the following persons: Gerhard Muenz for the
   thorough detail review and significant contribution regarding the
   improvement of whole sections; Daisuke Matsubara, Tsuyoshi Kondoh,
   Hiroshi Kurakami, Haruhiko Nishida for contribution during the
   initial phases of the document; Brian Trammel for contribution
   regarding the improvement of terminologies section; Nevil Brownlee,
   Juergen Quittek for the technical reviews and feedback.










































Kobayashi, et al.       Expires January 14, 2010               [Page 31]

Internet-Draft          IPFIX Mediation Framework              July 2009


Authors' Addresses

   Atsushi Kobayashi
   NTT Information Sharing Platform Laboratories
   3-9-11 Midori-cho
   Musashino-shi, Tokyo  180-8585
   Japan

   Phone: +81-422-59-3978
   Email: akoba@nttv6.net


   Benoit Claise
   Cisco Systems, Inc.
   De Kleetlaan 6a b1
   Diegem  1831
   Belgium

   Phone: +32 2 704 5622
   Email: bclaise@cisco.com


   Keisuke Ishibashi
   NTT Information Sharing Platform Laboratories
   3-9-11 Midori-cho
   Musashino-shi  180-8585
   Japan

   Phone: +81-422-59-3978
   Email: ishibashi.keisuke@lab.ntt.co.jp





















Kobayashi, et al.       Expires January 14, 2010               [Page 32]


Html markup produced by rfcmarkup 1.109, available from https://tools.ietf.org/tools/rfcmarkup/